Migrate existing Cypress tests to Cypress + Cucumber #57299

mdelapenya · 2020-02-11T10:24:40Z

Summary

It converts the existing Cypress tests for APM from Cypress to Cypress+Cucumber. To achieve that, it adds a cucumber-preprocessor, as shown in the following examples:

cypress + webpack + cucumber + typescript example: https://github.com/TheBrainFamily/cypress-cucumber-webpack-typescript-example
cypres + webpack example: https://github.com/cypress-io/cypress-example-recipes/tree/a240054d7f5626ffcd7bd668dded96d219c4a7eb/examples/preprocessors__typescript-webpack

The cypress tests are run using the same npm run cypress:run script, so no CI command should be affected.

To follow a proper Cypress layout, we enclosed the end-to-end (e2e) tests under the x-pack/legacy/plugins/apm/e2e directory, where its own package.json will be located, controlling the life cycle if these tests. Then, a cypress directory will contain the implementation of the e2e tests.

We've kept the buid dependencies under apm/e2e because we consider having the testime dependencies isolated from other plugins is important for the sake of running the test during local developement with a nice experience.

Checklist

~~- [ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support~~

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
~~- [ ] This was checked for keyboard-only and screenreader accessibility~~
~~- [ ] This renders correctly on smaller devices using a responsive layout. (You can test this in your browser~~
~~- [ ] This was checked for cross-browser compatibility, including a check against IE11~~

For maintainers

~~- [ ] This was checked for breaking API changes and was labeled appropriately~~

Steps to run the tests

Please follow instructions in the README of the APM UI tests:

Start Elasticsearch and APM Server, using apm-integration-testing:

$ git clone https://github.com/elastic/apm-integration-testing.git
$ cd apm-integration-testing
$ ./scripts/compose.py start master --no-kibana --no-xpack-secure

Download static data file

$ cd x-pack/legacy/plugins/apm/e2e/cypress/ingest-data
$ curl https://storage.googleapis.com/apm-ui-e2e-static-data/events.json --output events.json

Post to APM Server

$ cd x-pack/legacy/plugins/apm/e2e/cypress/ingest-data
$ node replay.js --server-url http://localhost:8200 --secret-token abcd --events ./events.json

This process will take a few minutes to ingest all data

Start Kibana

$ yarn kbn bootstrap
$ yarn start --no-base-path --csp.strict=false

Screenshots

Feature file

Tests passing locally in headless mode (`npm run cypress:run`):

Tests passing locally using Cypress app (`npm run cypress:open`):

Related Issues

Supersedes Migrate existing Cypress tests to Cypress + Cucumber #56970

elasticmachine · 2020-02-11T10:25:07Z

Pinging @elastic/apm-ui (Team:apm)

mdelapenya · 2020-02-11T10:27:28Z

.ci/end2end.groovy

@@ -94,7 +94,7 @@ pipeline {
        dir("${BASE_DIR}"){
          sh '''
            jobs -l
-            docker build --tag cypress ${CYPRESS_DIR}/ci
+            docker build --tag cypress --build-arg NODE_VERSION=$(cat .node-version) ${CYPRESS_DIR}/ci


Here we capture project's Node version from root directory

mdelapenya · 2020-02-11T10:32:16Z

x-pack/legacy/plugins/apm/e2e/README.md

+# Prepare and run Kibana locally
+$ x-pack/legacy/plugins/apm/e2e/ci/prepare-kibana.sh
+# Build Docker image for Kibana
+$ docker build --tag cypress --build-arg NODE_VERSION=$(cat .node-version) x-pack/legacy/plugins/apm/e2e/ci 
+# Run Docker image
+$ docker run --rm -t --user "$(id -u):$(id -g)" \
+    -v `pwd`:/app --network="host" \
+    --name cypress cypress


I verified that these steps works locally: the tests pass, although very slow

mdelapenya · 2020-02-11T10:35:42Z

x-pack/legacy/plugins/apm/e2e/cypress/integration/apm.feature

+
+  Background:
+    Given a user browses the APM UI application
+    When the user inspects the opbeans-go service


@MadameSheema how do you see having a When clause in a Background? I'm not completely sure if moving this line to the Scenario itself would be a more accurate approach. wdyt?

Hi!! Sorry for my late reply, I was a little bit busy. I don't like to use Backgrounds at all. This is because if you have a big file in a feature you can skip it when reading it.

A part from what I said in the previous comment. If you only have one scenario, it does not make sense to have a background at all :)

I was preparing for future updates... but I agree with you :) I'll send a commit moving the background step to the scenario. Thanks!!!

mdelapenya · 2020-02-11T10:36:20Z

x-pack/legacy/plugins/apm/e2e/package.json

@@ -5,17 +5,20 @@
  "license": "MIT",
  "scripts": {
    "cypress:open": "cypress open",
-    "cypress:run": "cypress run"
+    "cypress:run": "cypress run --spec **/*.feature"


We are keeping the build scripts using the installed Cypress

x-pack/legacy/plugins/apm/e2e/README.md

mdelapenya · 2020-02-18T16:50:46Z

After #57882 is merged, we can continue with this one, as the cypress+cucumber tests will not be type checked by upstream build.

It seems not relevant to the main purpose of these tests

See https://github.com/TheBrainFamily/cypress-cucumber-webpack-typescript-example

We are keeping cypress as rootDir to follow project's structure. On the other hand, having a second cypress directory at the 2nd level is the default structure, as shown in the examples: - https://github.com/TheBrainFamily/cypress-cucumber-webpack-typescript-example - https://github.com/cypress-io/cypress-example-recipes/tree/a240054d7f5626ffcd7bd668dded96d219c4a7eb/examples/preprocessors__typescript-webpack

It was automatically changed by tests

We are keeping the build system within the test application, isolating dependencies

mdelapenya · 2020-02-21T20:52:52Z

Hey @elastic/kibana-operations, is there anything we missed on our side here?

Thanks!

mdelapenya · 2020-02-26T08:49:50Z

@elasticmachine merge upstream

This would avoid not reading the background when the number of scenarios grows

kibanamachine · 2020-02-26T10:39:12Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 3d48f52

History

💚 Build #27826 succeeded cac807a
💚 Build #27822 succeeded 01f82a5
💛 Build #27598 was flaky fa992d0
💚 Build #27565 succeeded 0885d46
💔 Build #27551 failed 18ec45d

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

jbudz

ops files LGTM

* chore: move gitignore to the cypress directory * chore: ignore more test files * fix: do not check the Loading Message It seems not relevant to the main purpose of these tests * chore: use cypres + webpack + cucumber scaffolding See https://github.com/TheBrainFamily/cypress-cucumber-webpack-typescript-example * chore: add eslint and prettier for code linting * feat: convert existing Cypress test into BDD style * feat: add support for using proper Node version in MacOSX * chore: use tslint * chore: use old layout We are keeping cypress as rootDir to follow project's structure. On the other hand, having a second cypress directory at the 2nd level is the default structure, as shown in the examples: - https://github.com/TheBrainFamily/cypress-cucumber-webpack-typescript-example - https://github.com/cypress-io/cypress-example-recipes/tree/a240054d7f5626ffcd7bd668dded96d219c4a7eb/examples/preprocessors__typescript-webpack * chore: remove prelint script meanwhile we fix TS lint * chore: move test results to a specific directory * chore: rename variable following old code * chore: remove non-needed lints, as we are going to use kibana build * chore: import snapshot function from cypress * chore: add readFile utils back from a bad removal * chore: change format of JSON spec file It was automatically changed by tests * chore: move CI directory to the proper layout in order for Jenkins to work * chore: store test-results from proper dir on Jenkins * chore: store artifacts properly on Jenkins * Fix type issues * chore: rename test application to e2e (end-to-end) We are keeping the build system within the test application, isolating dependencies * docs: reorganise docs for APM UI e2e tests * fix: Use proper cypress support file * chore: use existing NPM script for running cypress on CI * chore: update paths in CI scripts * docs: document how the CI runs the tests * chore: use Node 10 for tests * chore: Use kibana's Node version for tests * chore: run yarn install * docs: update docs * fix: path was wrong * docs: fix paths and flags used to load data * docs: elasticsearch fix flag * docs: Bootstrap kibana before running it * docs: remove outdated info * chore: move background steps to the scenario This would avoid not reading the background when the number of scenarios grows Co-authored-by: Dario Gieselaar <[email protected]> Co-authored-by: Elastic Machine <[email protected]>

* chore: move gitignore to the cypress directory * chore: ignore more test files * fix: do not check the Loading Message It seems not relevant to the main purpose of these tests * chore: use cypres + webpack + cucumber scaffolding See https://github.com/TheBrainFamily/cypress-cucumber-webpack-typescript-example * chore: add eslint and prettier for code linting * feat: convert existing Cypress test into BDD style * feat: add support for using proper Node version in MacOSX * chore: use tslint * chore: use old layout We are keeping cypress as rootDir to follow project's structure. On the other hand, having a second cypress directory at the 2nd level is the default structure, as shown in the examples: - https://github.com/TheBrainFamily/cypress-cucumber-webpack-typescript-example - https://github.com/cypress-io/cypress-example-recipes/tree/a240054d7f5626ffcd7bd668dded96d219c4a7eb/examples/preprocessors__typescript-webpack * chore: remove prelint script meanwhile we fix TS lint * chore: move test results to a specific directory * chore: rename variable following old code * chore: remove non-needed lints, as we are going to use kibana build * chore: import snapshot function from cypress * chore: add readFile utils back from a bad removal * chore: change format of JSON spec file It was automatically changed by tests * chore: move CI directory to the proper layout in order for Jenkins to work * chore: store test-results from proper dir on Jenkins * chore: store artifacts properly on Jenkins * Fix type issues * chore: rename test application to e2e (end-to-end) We are keeping the build system within the test application, isolating dependencies * docs: reorganise docs for APM UI e2e tests * fix: Use proper cypress support file * chore: use existing NPM script for running cypress on CI * chore: update paths in CI scripts * docs: document how the CI runs the tests * chore: use Node 10 for tests * chore: Use kibana's Node version for tests * chore: run yarn install * docs: update docs * fix: path was wrong * docs: fix paths and flags used to load data * docs: elasticsearch fix flag * docs: Bootstrap kibana before running it * docs: remove outdated info * chore: move background steps to the scenario This would avoid not reading the background when the number of scenarios grows Co-authored-by: Dario Gieselaar <[email protected]> Co-authored-by: Elastic Machine <[email protected]> Co-authored-by: Manuel de la Peña <[email protected]> Co-authored-by: Dario Gieselaar <[email protected]> Co-authored-by: Elastic Machine <[email protected]>

mdelapenya requested review from a team as code owners February 11, 2020 10:24

mdelapenya added the Team:APM All issues that need APM UI Team support label Feb 11, 2020

mdelapenya added release_note:skip Skip the PR/issue when compiling release notes v7.6.0 labels Feb 11, 2020

mdelapenya requested a review from v1v February 11, 2020 10:26

mdelapenya commented Feb 11, 2020

View reviewed changes

v1v approved these changes Feb 14, 2020

View reviewed changes

x-pack/legacy/plugins/apm/e2e/README.md Outdated Show resolved Hide resolved

x-pack/legacy/plugins/apm/e2e/README.md Outdated Show resolved Hide resolved

mdelapenya force-pushed the cypress-cucumber branch from 361ddef to 29b45ac Compare February 18, 2020 17:32

mdelapenya added 16 commits February 19, 2020 09:05

chore: move gitignore to the cypress directory

3546da9

chore: ignore more test files

2151efe

fix: do not check the Loading Message

da613e6

It seems not relevant to the main purpose of these tests

chore: use cypres + webpack + cucumber scaffolding

29448f8

See https://github.com/TheBrainFamily/cypress-cucumber-webpack-typescript-example

chore: add eslint and prettier for code linting

da484fd

feat: convert existing Cypress test into BDD style

ea1ce45

feat: add support for using proper Node version in MacOSX

3fac1c9

chore: use tslint

a8fc985

chore: remove prelint script meanwhile we fix TS lint

7ecc7f5

chore: move test results to a specific directory

ccafe6e

chore: rename variable following old code

8ac1821

chore: remove non-needed lints, as we are going to use kibana build

aa13841

chore: import snapshot function from cypress

3ce17ca

chore: add readFile utils back from a bad removal

df48f2d

chore: change format of JSON spec file

cb50654

It was automatically changed by tests

dgieselaar and others added 11 commits February 19, 2020 09:05

Fix type issues

c6e6621

chore: rename test application to e2e (end-to-end)

ac0252b

We are keeping the build system within the test application, isolating dependencies

docs: reorganise docs for APM UI e2e tests

29033aa

fix: Use proper cypress support file

47b1d90

chore: use existing NPM script for running cypress on CI

3677779

chore: update paths in CI scripts

d73b62c

docs: document how the CI runs the tests

66e1656

chore: use Node 10 for tests

8bbe470

chore: Use kibana's Node version for tests

a729cde

chore: run yarn install

33be793

docs: update docs

18ec45d

mdelapenya force-pushed the cypress-cucumber branch from 29b45ac to 18ec45d Compare February 19, 2020 08:11

fix: path was wrong

0885d46

mdelapenya requested a review from sorenlouv February 19, 2020 10:26

mdelapenya added 4 commits February 19, 2020 13:42

docs: fix paths and flags used to load data

fa992d0

docs: elasticsearch fix flag

35f1795

Merge branch 'master' into cypress-cucumber

c6f6b9d

docs: Bootstrap kibana before running it

01f82a5

sorenlouv approved these changes Feb 20, 2020

View reviewed changes

docs: remove outdated info

cac807a

elasticmachine and others added 2 commits February 26, 2020 03:49

Merge branch 'master' into cypress-cucumber

067a7a2

chore: move background steps to the scenario

3d48f52

This would avoid not reading the background when the number of scenarios grows

jbudz approved these changes Feb 26, 2020

View reviewed changes

ogupte merged commit d27b6c0 into elastic:master Feb 26, 2020

ogupte mentioned this pull request Feb 26, 2020

[7.x] Migrate existing Cypress tests to Cypress + Cucumber (#57299) #58634

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate existing Cypress tests to Cypress + Cucumber #57299

Migrate existing Cypress tests to Cypress + Cucumber #57299

mdelapenya commented Feb 11, 2020 •

edited

Loading

elasticmachine commented Feb 11, 2020

mdelapenya Feb 11, 2020

mdelapenya Feb 11, 2020

mdelapenya Feb 11, 2020

MadameSheema Feb 24, 2020

MadameSheema Feb 24, 2020

mdelapenya Feb 26, 2020

mdelapenya Feb 11, 2020

mdelapenya commented Feb 18, 2020

mdelapenya commented Feb 21, 2020

mdelapenya commented Feb 26, 2020

kibanamachine commented Feb 26, 2020

jbudz left a comment

Migrate existing Cypress tests to Cypress + Cucumber #57299

Migrate existing Cypress tests to Cypress + Cucumber #57299

Conversation

mdelapenya commented Feb 11, 2020 • edited Loading

Summary

Checklist

For maintainers

Steps to run the tests

Screenshots

Feature file

Tests passing locally in headless mode (npm run cypress:run):

Tests passing locally using Cypress app (npm run cypress:open):

Related Issues

elasticmachine commented Feb 11, 2020

mdelapenya Feb 11, 2020

Choose a reason for hiding this comment

mdelapenya Feb 11, 2020

Choose a reason for hiding this comment

mdelapenya Feb 11, 2020

Choose a reason for hiding this comment

MadameSheema Feb 24, 2020

Choose a reason for hiding this comment

MadameSheema Feb 24, 2020

Choose a reason for hiding this comment

mdelapenya Feb 26, 2020

Choose a reason for hiding this comment

mdelapenya Feb 11, 2020

Choose a reason for hiding this comment

mdelapenya commented Feb 18, 2020

mdelapenya commented Feb 21, 2020

mdelapenya commented Feb 26, 2020

kibanamachine commented Feb 26, 2020

💚 Build Succeeded

History

jbudz left a comment

Choose a reason for hiding this comment

mdelapenya commented Feb 11, 2020 •

edited

Loading

Tests passing locally in headless mode (`npm run cypress:run`):

Tests passing locally using Cypress app (`npm run cypress:open`):