[APM] Migrate `/mobile` API tests to deployment agnostic folder #199021

kpatticha · 2024-11-05T16:20:57Z

closes #198980

In addition to migrating the mobile api tests, the PR includes

Fixing mapping issue with error.grouping_name which causing to drop documents
Fix and unskip mobile tests

How to test

Serverless

node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM"

It's recommended to be run against MKI

Stateful

node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts
node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM"

TODO

flaky runner
locally pass
mki run

Migrate agent_explorer test Update latest_agent_versions test

…fix'

elasticmachine · 2024-11-05T16:21:04Z

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

github-actions · 2024-11-05T16:21:13Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

dmlemeshko · 2024-11-06T10:12:32Z

I believe we can hold with this PR review until #198775 is merged?

kpatticha · 2024-11-06T10:26:40Z

@dmlemeshko yes definitely, I opened before seeing the discussion around the registry

…api-tests

dmlemeshko · 2024-11-07T06:11:43Z

...ntegration/deployment_agnostic/apis/observability/apm/mobile/errors/group_id_samples.spec.ts

+    // FLAKY: https://github.com/elastic/kibana/issues/177654
+    describe.skip('when samples data is loaded', () => {


question: do you think it can be safely migrated as "deployment-agnostic" in skip state without being run against MKI and ESS?

DO tests have pretty strict stability requirements: https://github.com/elastic/kibana/tree/main/x-pack/test/api_integration/deployment_agnostic#tagging-and-skipping-the-tests

I'm afraid that later you might face issues to re-enable due to those Cloud-first testing requirement

TBH, I don't think it's safe. I could try unskip these tests

…api-tests

miloszmarcinkowski

LGTM 🚀

...tegration/deployment_agnostic/apis/observability/apm/mobile/crashes/crash_group_list.spec.ts

dmlemeshko

LGTM, just few nits

...tegration/deployment_agnostic/apis/observability/apm/mobile/crashes/crash_group_list.spec.ts

...i_integration/deployment_agnostic/apis/observability/apm/mobile/crashes/distribution.spec.ts

...ntegration/deployment_agnostic/apis/observability/apm/mobile/errors/group_id_samples.spec.ts

...est/api_integration/deployment_agnostic/apis/observability/apm/mobile/mobile_filters.spec.ts

dmlemeshko · 2024-11-08T15:10:32Z

..._integration/deployment_agnostic/apis/observability/apm/mobile/mobile_location_stats.spec.ts

+            environment: 'production',
+            kuery: `service.version:"1.1"`,


is this configuration acceptable to run tests against ESS/MKI?

I notice tests use different query arguments and wonder if these tests will work not only in KIbana CI, but when we run it against serverless on MKI and 9.0.0-snapshot ESS

what do you mean?

I tested it against serverless on MKI and it worked.

we run all deployment-agnostic tests in Cloud both for serverless (MKI) and stateful (ESS 9.0.0-snapshot). I saw you marked MKI run checkbox, so my guess all mobile tests from PR are checked against MKI. Do we expect the same for ESS Cloud?

I run the tests

locally serverless

locally stateful

serverless MKI

do we also need to test against ESS cloud?

…a into migrate-mobile-api-tests

…-fix'

elasticmachine · 2024-11-11T13:09:25Z

⏳ Build in-progress

Buildkite Build
Commit: 59fd80c
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-199021-59fd80c5a595

History

💚 Build #249622 succeeded 3878972
💛 Build #249533 was flaky 995aaa9
💔 Build #249002 failed 0906fa1
💔 Build #248540 failed 4870a81

kibanamachine · 2024-11-11T13:50:13Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11779838721

kibanamachine · 2024-11-11T13:54:56Z

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts You might need to backport the following PRs to 8.x: - [Discover] Make use of stateContainer action onDataViewEdited (#199588) - [TSVB] Remove `metrics:allowCheckingForFailedShards` Advanced Setting (#197227) - [Security Solution] Rule Updates in bulk with conflicts (#196776) - [Search Playground] Change route on selector changed in between search/chat modes (#197431) - [OpenAPI][Fleet] Add missing operation summaries (#199548) - Update serverless link targets (#199396)

Manual backport

To create the backport manually run:

node scripts/backport --pr 199021

Questions ?

Please refer to the Backport tool documentation

kpatticha · 2024-11-11T18:47:11Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…tic#199021) closes elastic#198980 In addition to migrating the mobile api tests, the PR includes - Fixing mapping issue with `error.grouping_name` which causing to drop documents - Fix and unskip mobile tests ### How to test - Serverless ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM" ``` It's recommended to be run against [MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki) - Stateful ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM" ``` TODO - [x] flaky runner - [x] locally pass - [x] mki run --------- Co-authored-by: Carlos Crespo <[email protected]> Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Elastic Machine <[email protected]> (cherry picked from commit f77a805)

kibanamachine · 2024-11-12T14:00:10Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

…tic#199021) closes elastic#198980 In addition to migrating the mobile api tests, the PR includes - Fixing mapping issue with `error.grouping_name` which causing to drop documents - Fix and unskip mobile tests ### How to test - Serverless ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM" ``` It's recommended to be run against [MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki) - Stateful ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM" ``` TODO - [x] flaky runner - [x] locally pass - [x] mki run --------- Co-authored-by: Carlos Crespo <[email protected]> Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Elastic Machine <[email protected]>

kibanamachine · 2024-11-13T14:49:05Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

kibanamachine · 2024-11-14T15:49:13Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

kibanamachine · 2024-11-15T16:48:56Z

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

…#199021) (#199689) # Backport This will backport the following commits from `main` to `8.x`: - [[APM] Migrate `/mobile` API tests to deployment agnostic folder (#199021)](#199021)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)

…tic#199021) closes elastic#198980 In addition to migrating the mobile api tests, the PR includes - Fixing mapping issue with `error.grouping_name` which causing to drop documents - Fix and unskip mobile tests ### How to test - Serverless ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="APM" ``` It's recommended to be run against [MKI](https://github.com/crespocarlos/kibana/blob/main/x-pack/test_serverless/README.md#run-tests-on-mki) - Stateful ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="APM" ``` TODO - [x] flaky runner - [x] locally pass - [x] mki run --------- Co-authored-by: Carlos Crespo <[email protected]> Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Elastic Machine <[email protected]>

crespocarlos and others added 8 commits November 4, 2024 14:26

Add foundation to run apm tests

eee7cd3

Migrate agent_explorer test Update latest_agent_versions test

Force Git to recognize latest_agent_versions.spec.ts as moved

5857da2

Updated latest_agent_versions.spec.ts after moving

c45c662

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

66f5e4c

…fix'

Simplify apmApi

17fb5e2

Symplify createApmSynthtraceEsClient

4d684c6

Merge branch 'main' into 193245-apm-deployment-agnostic-tests-part-1

06e7f74

[APM] Migrate /mobile API tests to deployment agnostic folder

4870a81

kpatticha requested review from a team as code owners November 5, 2024 16:20

botelastic bot added ci:project-deploy-observability Create an Observability project Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team labels Nov 5, 2024

kpatticha changed the title ~~[APM] Migrate /mobile API tests to deployment agnostic folder~~ [APM] Migrate /mobile API tests to deployment agnostic folder Nov 5, 2024

kpatticha added release_note:skip Skip the PR/issue when compiling release notes backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) labels Nov 5, 2024

kpatticha added 5 commits November 6, 2024 20:08

Merge branch 'main' of github.com:elastic/kibana into migrate-mobile-…

4671875

…api-tests

Remove registry service

9a19a9b

fix

a1db7f1

Remove registry service

1d64be8

remove service once and for all

0906fa1

dmlemeshko reviewed Nov 7, 2024

View reviewed changes

kpatticha added 4 commits November 8, 2024 13:42

Unskip mobile tests

14105cb

Fix synthrace dropped document due to wrong type

c9bccbf

Unskip tests

0f511a1

Merge branch 'main' of github.com:elastic/kibana into migrate-mobile-…

6b4ddc0

…api-tests

kpatticha requested a review from a team as a code owner November 8, 2024 11:55

miloszmarcinkowski approved these changes Nov 8, 2024

View reviewed changes

...tegration/deployment_agnostic/apis/observability/apm/mobile/crashes/crash_group_list.spec.ts Outdated Show resolved Hide resolved

Clean up code

06403c2

dmlemeshko approved these changes Nov 8, 2024

View reviewed changes

Merge branch 'migrate-mobile-api-tests' of github.com:kpatticha/kiban…

3878972

…a into migrate-mobile-api-tests

kpatticha mentioned this pull request Nov 8, 2024

[APM] Migrate /test/apm_api_integration/tests/errors/ to be deployment-agnostic API tests #198971

Closed

Merge branch 'main' into migrate-mobile-api-tests

76d5c9f

kpatticha enabled auto-merge (squash) November 11, 2024 11:41

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

59fd80c

…-fix'

kpatticha merged commit f77a805 into elastic:main Nov 11, 2024
27 checks passed

kibanamachine added the v9.0.0 label Nov 11, 2024

kpatticha mentioned this pull request Nov 11, 2024

[8.x] [APM] Migrate /mobile API tests to deployment agnostic folder (#199021) #199689

Merged

kibanamachine mentioned this pull request Nov 12, 2024

[APM] Migrate apm alerts tests to deployment agnostic #199097

Merged

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Nov 12, 2024

kibanamachine mentioned this pull request Nov 12, 2024

[APM] Migrate /environment to deployment agnostic test #199582

Merged

4 tasks

miloszmarcinkowski mentioned this pull request Nov 13, 2024

[APM] Migrate test apm-api-integration/tests/errors to be deployment agnostic api tests #199655

Merged

kibanamachine mentioned this pull request Nov 18, 2024

Migrate /test/apm_api_integration/tests/span_links to be deployment-agnostic API tests #200140

Merged

4 tasks

kibanamachine added v8.17.0 and removed backport missing Added to PRs automatically when the are determined to be missing a backport. labels Nov 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[APM] Migrate `/mobile` API tests to deployment agnostic folder #199021

[APM] Migrate `/mobile` API tests to deployment agnostic folder #199021

kpatticha commented Nov 5, 2024 •

edited by kibanamachine

Loading

elasticmachine commented Nov 5, 2024

github-actions bot commented Nov 5, 2024

dmlemeshko commented Nov 6, 2024

kpatticha commented Nov 6, 2024

dmlemeshko Nov 7, 2024 •

edited

Loading

kpatticha Nov 7, 2024

miloszmarcinkowski left a comment

dmlemeshko left a comment

dmlemeshko Nov 8, 2024

dmlemeshko Nov 8, 2024 •

edited

Loading

kpatticha Nov 8, 2024

kpatticha Nov 8, 2024

dmlemeshko Nov 8, 2024 •

edited

Loading

kpatticha Nov 8, 2024

elasticmachine commented Nov 11, 2024

kibanamachine commented Nov 11, 2024

kibanamachine commented Nov 11, 2024

kpatticha commented Nov 11, 2024

kibanamachine commented Nov 12, 2024

kibanamachine commented Nov 13, 2024

kibanamachine commented Nov 14, 2024

kibanamachine commented Nov 15, 2024

		// FLAKY: https://github.com/elastic/kibana/issues/177654
		describe.skip('when samples data is loaded', () => {

[APM] Migrate /mobile API tests to deployment agnostic folder #199021

[APM] Migrate /mobile API tests to deployment agnostic folder #199021

Conversation

kpatticha commented Nov 5, 2024 • edited by kibanamachine Loading

How to test

elasticmachine commented Nov 5, 2024

github-actions bot commented Nov 5, 2024

🤖 GitHub comments

dmlemeshko commented Nov 6, 2024

kpatticha commented Nov 6, 2024

dmlemeshko Nov 7, 2024 • edited Loading

Choose a reason for hiding this comment

kpatticha Nov 7, 2024

Choose a reason for hiding this comment

miloszmarcinkowski left a comment

Choose a reason for hiding this comment

dmlemeshko left a comment

Choose a reason for hiding this comment

dmlemeshko Nov 8, 2024

Choose a reason for hiding this comment

dmlemeshko Nov 8, 2024 • edited Loading

Choose a reason for hiding this comment

kpatticha Nov 8, 2024

Choose a reason for hiding this comment

kpatticha Nov 8, 2024

Choose a reason for hiding this comment

dmlemeshko Nov 8, 2024 • edited Loading

Choose a reason for hiding this comment

kpatticha Nov 8, 2024

Choose a reason for hiding this comment

elasticmachine commented Nov 11, 2024

⏳ Build in-progress

History

kibanamachine commented Nov 11, 2024

kibanamachine commented Nov 11, 2024

💔 All backports failed

Manual backport

Questions ?

kpatticha commented Nov 11, 2024

💚 All backports created successfully

Questions ?

kibanamachine commented Nov 12, 2024

kibanamachine commented Nov 13, 2024

kibanamachine commented Nov 14, 2024

kibanamachine commented Nov 15, 2024

[APM] Migrate `/mobile` API tests to deployment agnostic folder #199021

[APM] Migrate `/mobile` API tests to deployment agnostic folder #199021

kpatticha commented Nov 5, 2024 •

edited by kibanamachine

Loading

dmlemeshko Nov 7, 2024 •

edited

Loading

dmlemeshko Nov 8, 2024 •

edited

Loading

dmlemeshko Nov 8, 2024 •

edited

Loading