[APM] 8.5.0 Test Plan

[dannycroft]

Test Plan 8.5.0

Phase 1 - Manual Testing: Monday 26th September

• Pick an issue from the curated list
• After testing apply apm:test-plan-done label. If a problem is found a new issue should be opened and labelled apm:test-plan-regression and moved to "Ready" column.

Run latest build candidate (BC)

Using apm-integration-testing it's trivial to start the stack for the latest BC:

./scripts/compose.py start 8.5.0 --bc --all-opbeans

Check to see if issues are fixed before opening a bug

Find the commit kibana was built with:

./scripts/compose.py versions
...

Kibana (image built: 2022-09-22 07:33:44 UTC):
	Version: 8.5.0-SNAPSHOT
	Branch: 8.5
	Build SHA: 0d8de4df69f8084a94cdd9638d7de510813cb5ce
	Build number: 6678

Using that Build SHA, visit 0d8de4d...8.5 to see commits since the BC was built.

Creating users locally

node ./x-pack/plugins/apm/scripts/create_apm_users.js --username elastic --password changeme --kibana-url http://localhost:5601

Creates the following users (username / password):

• viewer / changeme
• editor / changeme

Phase 2 - Improving Quality: Tuesday 27th September - Friday 30th September

Primarily we need more e2e (cypress) tests and api tests. In addition to this we should replace the archives containing random opbeans data with synthbeans generated data. Below is a list of the important flows that should be covered in both a unit and an api test.

Important areas & flows

• [APM] Storage Explorer Beta #137084
• [APM] Service Groups Beta: Dynamic Refresh #133112
• [APM] Add Experimental Mode for APM UI #138394
• [APM] Expose K8s meta data in the service overview page #138399
• [APM] Add AWS Lambda metrics to "Metrics" tab #138420
• [APM] Support Service Metrics on the inventory overview page #138527

Migrate from opbeans archives to synthtrace scenarios

We should replace the existing opbeans-based archives with data generated at runtime with synthtrace.

Migrate API test to apmApiClient

API tests should use the new apmApiClient instead of the legacy supertest clients. Example on how to migrate.

[elasticmachine]

Pinging @elastic/apm-ui (Team:APM)

[yngrdyn]

We have migrated the API tests to apmApiClient.
For more info check 142229 and 142724.

[sorenlouv]

Awesome @yngrdyn . Milestone achieved!