Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate existing Cypress tests to Cypress + Cucumber #57299

Merged
merged 38 commits into from
Feb 26, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
3546da9
chore: move gitignore to the cypress directory
mdelapenya Feb 3, 2020
2151efe
chore: ignore more test files
mdelapenya Feb 3, 2020
da613e6
fix: do not check the Loading Message
mdelapenya Feb 3, 2020
29448f8
chore: use cypres + webpack + cucumber scaffolding
mdelapenya Feb 3, 2020
da484fd
chore: add eslint and prettier for code linting
mdelapenya Feb 3, 2020
ea1ce45
feat: convert existing Cypress test into BDD style
mdelapenya Feb 4, 2020
3fac1c9
feat: add support for using proper Node version in MacOSX
mdelapenya Feb 4, 2020
a8fc985
chore: use tslint
mdelapenya Feb 4, 2020
51451a7
chore: use old layout
mdelapenya Feb 4, 2020
7ecc7f5
chore: remove prelint script meanwhile we fix TS lint
mdelapenya Feb 4, 2020
ccafe6e
chore: move test results to a specific directory
mdelapenya Feb 4, 2020
8ac1821
chore: rename variable following old code
mdelapenya Feb 4, 2020
aa13841
chore: remove non-needed lints, as we are going to use kibana build
mdelapenya Feb 6, 2020
3ce17ca
chore: import snapshot function from cypress
mdelapenya Feb 6, 2020
df48f2d
chore: add readFile utils back from a bad removal
mdelapenya Feb 6, 2020
cb50654
chore: change format of JSON spec file
mdelapenya Feb 6, 2020
d1b3cf3
chore: move CI directory to the proper layout in order for Jenkins to…
mdelapenya Feb 6, 2020
5e573fb
chore: store test-results from proper dir on Jenkins
mdelapenya Feb 6, 2020
a640d1d
chore: store artifacts properly on Jenkins
mdelapenya Feb 6, 2020
c6e6621
Fix type issues
dgieselaar Feb 7, 2020
ac0252b
chore: rename test application to e2e (end-to-end)
mdelapenya Feb 10, 2020
29033aa
docs: reorganise docs for APM UI e2e tests
mdelapenya Feb 10, 2020
47b1d90
fix: Use proper cypress support file
mdelapenya Feb 10, 2020
3677779
chore: use existing NPM script for running cypress on CI
mdelapenya Feb 10, 2020
d73b62c
chore: update paths in CI scripts
mdelapenya Feb 11, 2020
66e1656
docs: document how the CI runs the tests
mdelapenya Feb 11, 2020
8bbe470
chore: use Node 10 for tests
mdelapenya Feb 11, 2020
a729cde
chore: Use kibana's Node version for tests
mdelapenya Feb 11, 2020
33be793
chore: run yarn install
mdelapenya Feb 11, 2020
18ec45d
docs: update docs
mdelapenya Feb 11, 2020
0885d46
fix: path was wrong
mdelapenya Feb 19, 2020
fa992d0
docs: fix paths and flags used to load data
mdelapenya Feb 19, 2020
35f1795
docs: elasticsearch fix flag
mdelapenya Feb 19, 2020
c6f6b9d
Merge branch 'master' into cypress-cucumber
mdelapenya Feb 20, 2020
01f82a5
docs: Bootstrap kibana before running it
mdelapenya Feb 20, 2020
cac807a
docs: remove outdated info
mdelapenya Feb 20, 2020
067a7a2
Merge branch 'master' into cypress-cucumber
elasticmachine Feb 26, 2020
3d48f52
chore: move background steps to the scenario
mdelapenya Feb 26, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .ci/end2end.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ pipeline {
BASE_DIR = 'src/github.com/elastic/kibana'
HOME = "${env.WORKSPACE}"
APM_ITS = 'apm-integration-testing'
CYPRESS_DIR = 'x-pack/legacy/plugins/apm/cypress'
CYPRESS_DIR = 'x-pack/legacy/plugins/apm/e2e'
PIPELINE_LOG_LEVEL = 'DEBUG'
}
options {
Expand Down Expand Up @@ -107,7 +107,7 @@ pipeline {
dir("${BASE_DIR}"){
sh '''
jobs -l
docker build --tag cypress ${CYPRESS_DIR}/ci
docker build --tag cypress --build-arg NODE_VERSION=$(cat .node-version) ${CYPRESS_DIR}/ci
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we capture project's Node version from root directory

docker run --rm -t --user "$(id -u):$(id -g)" \
-v `pwd`:/app --network="host" \
--name cypress cypress'''
Expand All @@ -116,8 +116,8 @@ pipeline {
post {
always {
dir("${BASE_DIR}"){
archiveArtifacts(allowEmptyArchive: false, artifacts: "${CYPRESS_DIR}/screenshots/**,${CYPRESS_DIR}/videos/**,${CYPRESS_DIR}/*e2e-tests.xml")
junit(allowEmptyResults: true, testResults: "${CYPRESS_DIR}/*e2e-tests.xml")
archiveArtifacts(allowEmptyArchive: false, artifacts: "${CYPRESS_DIR}/**/screenshots/**,${CYPRESS_DIR}/**/videos/**,${CYPRESS_DIR}/**/test-results/*e2e-tests.xml")
junit(allowEmptyResults: true, testResults: "${CYPRESS_DIR}/**/test-results/*e2e-tests.xml")
}
dir("${APM_ITS}"){
sh 'docker-compose logs > apm-its.log || true'
Expand Down
2 changes: 1 addition & 1 deletion .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ src/legacy/core_plugins/vis_type_vislib/public/vislib/__tests__/lib/fixtures/moc
/x-pack/legacy/plugins/infra/common/graphql/types.ts
/x-pack/legacy/plugins/infra/public/graphql/types.ts
/x-pack/legacy/plugins/infra/server/graphql/types.ts
/x-pack/legacy/plugins/apm/cypress/**/snapshots.js
/x-pack/legacy/plugins/apm/e2e/cypress/**/snapshots.js
/src/legacy/plugin_discovery/plugin_pack/__tests__/fixtures/plugins/broken
**/graphql/types.ts
**/*.js.snap
Expand Down
2 changes: 1 addition & 1 deletion src/cli/cluster/cluster_manager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,7 @@ export class ClusterManager {
fromRoot('src/legacy/server/sass/__tmp__'),
fromRoot('x-pack/legacy/plugins/reporting/.chromium'),
fromRoot('x-pack/legacy/plugins/siem/cypress'),
fromRoot('x-pack/legacy/plugins/apm/cypress'),
fromRoot('x-pack/legacy/plugins/apm/e2e/cypress'),
fromRoot('x-pack/legacy/plugins/apm/scripts'),
fromRoot('x-pack/legacy/plugins/canvas/canvas_plugin_src'), // prevents server from restarting twice for Canvas plugin changes,
'plugins/java_languageserver',
Expand Down
2 changes: 1 addition & 1 deletion src/dev/ci_setup/setup_env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -168,4 +168,4 @@ if [[ -d "$ES_DIR" && -f "$ES_JAVA_PROP_PATH" ]]; then
export JAVA_HOME=$HOME/.java/$ES_BUILD_JAVA
fi

export CI_ENV_SETUP=true
export CI_ENV_SETUP=true
2 changes: 1 addition & 1 deletion src/dev/run_check_lockfile_symlinks.js
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ const IGNORE_FILE_GLOBS = [
// fixtures aren't used in production, ignore them
'**/*fixtures*/**/*',
// cypress isn't used in production, ignore it
'x-pack/legacy/plugins/apm/cypress/*',
'x-pack/legacy/plugins/apm/e2e/*',
];

run(async ({ log }) => {
Expand Down
2 changes: 1 addition & 1 deletion src/dev/typescript/projects.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ export const PROJECTS = [
new Project(resolve(REPO_ROOT, 'x-pack/legacy/plugins/siem/cypress/tsconfig.json'), {
name: 'siem/cypress',
}),
new Project(resolve(REPO_ROOT, 'x-pack/legacy/plugins/apm/cypress/tsconfig.json'), {
new Project(resolve(REPO_ROOT, 'x-pack/legacy/plugins/apm/e2e/tsconfig.json'), {
name: 'apm/cypress',
disableTypeCheck: true,
}),
Expand Down
63 changes: 0 additions & 63 deletions x-pack/legacy/plugins/apm/cypress/README.md

This file was deleted.

18 changes: 0 additions & 18 deletions x-pack/legacy/plugins/apm/cypress/cypress.json

This file was deleted.

53 changes: 0 additions & 53 deletions x-pack/legacy/plugins/apm/cypress/integration/apm.spec.ts

This file was deleted.

1 change: 0 additions & 1 deletion x-pack/legacy/plugins/apm/cypress/screenshots/.gitignore

This file was deleted.

3 changes: 0 additions & 3 deletions x-pack/legacy/plugins/apm/cypress/snapshots.js

This file was deleted.

10 changes: 0 additions & 10 deletions x-pack/legacy/plugins/apm/cypress/support/index.ts

This file was deleted.

4 changes: 4 additions & 0 deletions x-pack/legacy/plugins/apm/e2e/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
cypress/ingest-data/events.json
cypress/screenshots/*

cypress/test-results
68 changes: 68 additions & 0 deletions x-pack/legacy/plugins/apm/e2e/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# End-To-End (e2e) Test for APM UI

## Ingest static data into Elasticsearch via APM Server

1. Start Elasticsearch and APM Server, using [apm-integration-testing](https://github.com/elastic/apm-integration-testing):

```shell
$ git clone https://github.com/elastic/apm-integration-testing.git
$ cd apm-integration-testing
./scripts/compose.py start master --no-kibana --no-xpack-secure
```

2. Download [static data file](https://storage.googleapis.com/apm-ui-e2e-static-data/events.json)

```shell
$ cd x-pack/legacy/plugins/apm/e2e/cypress/ingest-data
$ curl https://storage.googleapis.com/apm-ui-e2e-static-data/events.json --output events.json
```

3. Post to APM Server

```shell
$ cd x-pack/legacy/plugins/apm/e2e/cypress/ingest-data
$ node replay.js --server-url http://localhost:8200 --secret-token abcd --events ./events.json
```
>This process will take a few minutes to ingest all data

4. Start Kibana

```shell
$ yarn kbn bootstrap
$ yarn start --no-base-path --csp.strict=false
```

> Content Security Policy (CSP) Settings: Your Kibana instance must have the `csp.strict: false`.

## How to run the tests

_Note: Run the following commands from `kibana/x-pack/legacy/plugins/apm/e2e/cypress`._

### Interactive mode

```
yarn cypress open
```

### Headless mode

```
yarn cypress run
```

## Reproducing CI builds

>This process is very slow compared to the local development described above. Consider that the CI must install and configure the build tools and create a Docker image for the project to run tests in a consistent manner.

The Jenkins CI uses a shell script to prepare Kibana:

```shell
# Prepare and run Kibana locally
$ x-pack/legacy/plugins/apm/e2e/ci/prepare-kibana.sh
# Build Docker image for Kibana
$ docker build --tag cypress --build-arg NODE_VERSION=$(cat .node-version) x-pack/legacy/plugins/apm/e2e/ci
# Run Docker image
$ docker run --rm -t --user "$(id -u):$(id -g)" \
-v `pwd`:/app --network="host" \
--name cypress cypress
Comment on lines +60 to +67
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified that these steps works locally: the tests pass, although very slow
Screenshot 2020-02-11 at 11 31 53

```
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
FROM node:12
ARG NODE_VERSION
FROM node:$NODE_VERSION

RUN apt-get -qq update \
&& apt-get -y -qq install xvfb \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ npm config set cache ${HOME}
# --exclude=packages/ \
# --exclude=built_assets --exclude=target \
# --exclude=data /app ${HOME}/
#cd ${HOME}/app/x-pack/legacy/plugins/apm/cypress
#cd ${HOME}/app/x-pack/legacy/plugins/apm/e2e/cypress

cd /app/x-pack/legacy/plugins/apm/cypress
cd /app/x-pack/legacy/plugins/apm/e2e
## Install dependencies for cypress
CI=true npm install
yarn install
Expand All @@ -33,4 +33,4 @@ npm install wait-on
./node_modules/.bin/wait-on ${CYPRESS_BASE_URL}/status && echo 'Kibana is up and running'

# Run cypress
./node_modules/.bin/cypress run
npm run cypress:run
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
set -e

CYPRESS_DIR="x-pack/legacy/plugins/apm/cypress"
CYPRESS_DIR="x-pack/legacy/plugins/apm/e2e"

echo "1/3 Install dependencies ..."
# shellcheck disable=SC1091
Expand Down
19 changes: 19 additions & 0 deletions x-pack/legacy/plugins/apm/e2e/cypress.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"baseUrl": "http://localhost:5601",
"video": false,
"trashAssetsBeforeRuns": false,
"fileServerFolder": "../",
"fixturesFolder": "./cypress/fixtures",
"integrationFolder": "./cypress/integration",
"pluginsFile": "./cypress/plugins/index.js",
"screenshotsFolder": "./cypress/screenshots",
"supportFile": "./cypress/support/index.ts",
"videosFolder": "./cypress/videos",
"useRelativeSnapshots": true,
"reporter": "junit",
"reporterOptions": {
"mochaFile": "./cypress/test-results/[hash]-e2e-tests.xml",
"toConsole": false
},
"testFiles": "**/*.{feature,features}"
}
7 changes: 7 additions & 0 deletions x-pack/legacy/plugins/apm/e2e/cypress/integration/apm.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Feature: APM

Scenario: Transaction duration charts
Given a user browses the APM UI application
When the user inspects the opbeans-go service
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MadameSheema how do you see having a When clause in a Background? I'm not completely sure if moving this line to the Scenario itself would be a more accurate approach. wdyt?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi!! Sorry for my late reply, I was a little bit busy. I don't like to use Backgrounds at all. This is because if you have a big file in a feature you can skip it when reading it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A part from what I said in the previous comment. If you only have one scenario, it does not make sense to have a background at all :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was preparing for future updates... but I agree with you :) I'll send a commit moving the background step to the scenario. Thanks!!!

Then should redirect to correct path with correct params
And should have correct y-axis ticks
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,12 @@ module.exports = {
}
}
},
"__version": "3.4.1"
"__version": "3.8.3",
"APM": {
"Transaction duration charts": {
"1": "3.7 min",
"2": "1.8 min",
"3": "0.0 min"
}
}
}
Loading