[Lens] Add serverless functional tests

[nickofthyme]

Summary

This PR copies, with limited modifications, functional test suites from test/functional and x-pack/test/functional

Lens Smoke tests

• Smokescreen tests (x-pack/test/functional/apps/lens/group1/smokescreen.ts)
• Basic vega tests (test/functional/apps/visualize/group6/_vega_chart.ts)
• Sanity checks for TSDB functionality in Lens x-pack/test/functional/apps/lens/group4/tsdb.ts

Convert to Lens

These tests outside of serverless, assume the following vis types are accessible via the Visualize editor, however all the following types are designated readonly in serverless and thus the fn tests fail.

In place of these tests I created the identical visualizations, added them to a dashboard and exported the SOs. These dashboard and dependent visualizations are loaded in each respective test, and the assertions are run on the Convert to Lens option from the dashboard. All assertions are mostly unchanged or achieve the same effect.

• ✅ Agg based - x-pack/test/functional/apps/lens/open_in_lens/agg_based
• ✅ TSVB - x-pack/test/functional/apps/lens/open_in_lens/tsvb
• ❌ Dashboard - x-pack/test/functional/apps/lens/open_in_lens/dashboard/config.ts
  • Not applicable to serverless env and/or duplicate of other tests.

Closes #162346

[Ikuni17]

OPs changes LGTM. Please hold off on merging this until #162673 has been merged though (should be today). Then merge main and we can test against a full serverless environment. Thanks!

[pheyos]

@nickofthyme @stratoula serverless common config grouping is merged now (#167172).

With that, you can add more common config files to x-pack/test_serverless/functional/test_suites/[observability|search|security]/common_configs and include proper chunks of your added common tests, so run time of each config is ideally less than ~10-15 minutes.

Don't forget to also add your config files to .buildkite/ftr_configs.yml.

[nickofthyme]

@stratoula I moved the visualizations/ tests into their own config files and still get the kibanaChrome error.

1) Visualizations

  lens serverless - group 1
  lens smokescreen tests
  should allow creation of lens xy chart:
   
  Error: retry.tryForTime timeout: Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="kibanaChrome"])
  Wait timed out after 61118ms
  at /var/lib/buildkite-agent/builds/kb-n2-4-spot-39f9407726eaba6f/elastic/kibana-pull-request/kibana/node_modules/selenium-webdriver/lib/webdriver.js:929:17
  at processTicksAndRejections (node:internal/process/task_queues:95:5)
  at onFailure (retry_for_success.ts:17:9)
  at retryForSuccess (retry_for_success.ts:59:13)
  at RetryService.try (retry.ts:31:12)
  at common_page.ts:273:21
  at runAttempt (retry_for_success.ts:29:15)
  at retryForSuccess (retry_for_success.ts:68:21)
  at RetryService.tryForTime (retry.ts:22:12)
  at CommonPageObject.navigateToApp (common_page.ts:272:5)
  at VisualizePageObject.gotoVisualizationLandingPage (visualize_page.ts:69:5)
  at VisualizePageObject.navigateToNewVisualization (visualize_page.ts:145:5)
  at Context.<anonymous> (smokescreen.ts:24:7)
  at Object.apply (wrap_function.js:73:16)
  at onFailure (retry_for_success.ts:17:9)
  at retryForSuccess (retry_for_success.ts:59:13)
  at RetryService.tryForTime (retry.ts:22:12)
  at CommonPageObject.navigateToApp (common_page.ts:272:5)
  at VisualizePageObject.gotoVisualizationLandingPage (visualize_page.ts:69:5)
  at VisualizePageObject.navigateToNewVisualization (visualize_page.ts:145:5)
  at Context.<anonymous> (smokescreen.ts:24:7)
  at Object.apply (wrap_function.js:73:16)

Looks like there are 33 issues of failed tests with similar issues here. Have you heard anything about this? The error screenshots point to an authentication problem.

[stratoula]

No 🤔 @pheyos or maybe @davismcphee do you have any idea which is the problem here?

[davismcphee]

@nickofthyme @stratoula If it's failing consistently it's possible there's a missing call to await PageObjects.svlCommonPage.login(); in the test. But if it's failing intermittently, it's likely the same cause as other tests, Serverless CI instability. I think the hope was that this PR would resolve the issue in CI, but it seems we're still seeing the failures: #166936.

[pheyos]

@davismcphee @nickofthyme @stratoula there was an issue on the Elasticsearch side that caused the login/logout flakiness in serverless. A fix is merged now on their end and during first checks on the flaky test runner, the flakiness didn't come up again. So we should hopefully be good now 🤞

[kibana-ci]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 075a542

Failed CI Steps

• FTR Configs #16

Test Failures

• [job] [logs] FTR Configs #16 / AIOps POST /internal/aiops/log_rate_analysis - groups only with ecommerce should return group only in chunks with streaming without compression without flushFix

Metrics [docs]

✅ unchanged

History

• 💔 Build #165386 failed e27c9ec
• 💔 Build #164869 failed d21b15d
• 💔 Build #164833 failed 7318b4c
• 💔 Build #164760 failed 4da1af2
• 💔 Build #164243 failed 151bd38

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream