Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Functional tests - add basic license tests #82147

Merged
merged 4 commits into from
Nov 2, 2020

Conversation

pheyos
Copy link
Member

@pheyos pheyos commented Oct 30, 2020

Summary

This PR adds initial functional tests and api integration tests running on a basic license for the ML and Transform apps.

Details

Directory structure / CI

As discussed with @spalger and @dmlemeshko, in x-pack/test there are now two new directories: api_integration_basic and functional_basic that reuse esArchives, services, pageObjects and configs from api_integration and functional respectively, basically just overriding the license and the test report name. This way it's easy to reuse in case other apps want to add basic license tests as well in the future without the overhead of additional configs.

image
image

The new directories are added to the functional_tests script file and the suites will run in ciGroup2 for now (since as of writing this, that group has the shortest runtime).

Added basic license tests

Some existing (trial license) test files are included for the features that should work the same on a basic license.

ML:

  • api_integration_basic
    • Include the existing (trial license) data_visualizer tests
  • functional_basic
    • Include the existing (trial license) data_visualizer tests with the exception of the advanced job wizard test, see below in Other changes
    • Add permission tests similar to the existing (trial license) ones, but adjusted for validating the UI on basic license

Transform:

  • Transform is a basic license feature, so all existing api_integration and functional tests are included and nothing is added.

Other changes

  • Move the index data visualizer open advanced wizard test to a separate test file to make the rest of the tests reusable for basic license
  • Check enabled / disabled ML tabs (trial + basic)
  • Check the data visualizer start trial card
  • Don't expect the overview tab as landing page when navigating to ML (in basic it's the data visualizer tab), instead check for the mlApp to exist.

Run the tests

From the x-pack directory run

node scripts/functional_tests_server.js --config test/functional_basic/config.ts
node scripts/functional_test_runner.js --config test/functional_basic/config.ts

for the functional basic license tests or

node scripts/functional_tests_server.js --config test/api_integration_basic/config.ts
node scripts/functional_test_runner.js --config test/api_integration_basic/config.ts

for the api integration basic license tests.

@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@pheyos
Copy link
Member Author

pheyos commented Nov 2, 2020

Checking test stability in a flaky test runner job ...
No ML / Transform failure in 50 runs ✔️ (just one unrelated task manager test failure)

Copy link
Contributor

@peteharverson peteharverson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just spotted a few typos

Copy link
Contributor

@peteharverson peteharverson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spotted another couple of typos

Copy link
Contributor

@peteharverson peteharverson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding these basic license tests in! Tested locally and LGTM.

Copy link
Contributor

@darnautov darnautov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

async chunks size

id before after diff
ml 6.6MB 6.6MB +312.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@spalger spalger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pheyos pheyos merged commit 45667d0 into elastic:master Nov 2, 2020
@pheyos pheyos deleted the basic_license_tests branch November 2, 2020 16:30
pheyos added a commit to pheyos/kibana that referenced this pull request Nov 2, 2020
This PR adds initial functional tests and api integration tests running on a basic license for the ML and Transform apps.
pheyos added a commit that referenced this pull request Nov 3, 2020
This PR adds initial functional tests and api integration tests running on a basic license for the ML and Transform apps.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants