Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM] Toggle Column / Code Coverage and Cypress #42766

Merged

Conversation

andrew-goldstein
Copy link
Contributor

@andrew-goldstein andrew-goldstein commented Aug 6, 2019

[SIEM] Toggle Column / Code Coverage and Cypress

Features

  • A Toggle column checkbox for adding / removing fields in the timeline offers
    an alternative to drag and drop in an expanded event:

toggle-field

The behavior of the Toggle field checkbox in the expanded event is the same
as the checkboxes that appear in the timeline Fields Browser.

Fixes

  • Pinned timeline events now use a "filled" styling when pinned instead of rotation, and unpinned events have a higher contrast in dark mode:

Before:

unpinned-before

After (unpinned):
after-unpinned

After (pinned):
after-pinned

  • Fixed an issue where the Copy to Clipboard icon rendered incorrectly (too
    small, and not the right color):

Before:

copy-to-clipboard-before

After:

copy-to-clipboard-after

  • Fixed an issue where a previously invisible control became visible and
    occupied additonal row space in the timeline:

Before:

row-height-before

After:

row-height-after

  • Fixed an issue where the timeline Fields Browser Overlaps Column Headers

Before:

field-browser-position-before

After:

field-browser-position-after

Code Coverage and Cypress Tests

  • This PR increases the jest unit test coverage for the Fields Browser, and
    introduces Cypress tests for it
  • Added new Cypress test helpers, configuration, and test refactorings to
    support running smoke tests against remote Kibana instances, including
    Elastic Cloud
  • Refactored some Cypress tests to speed them up and reduce flakyness

https://github.com/elastic/siem-team/issues/380
https://github.com/elastic/siem-team/issues/431
https://github.com/elastic/siem-team/issues/432
https://github.com/elastic/siem-team/issues/433
https://github.com/elastic/siem-team/issues/434
https://github.com/elastic/siem-team/issues/435
https://github.com/elastic/siem-team/issues/436

@elasticmachine
Copy link
Contributor

Pinging @elastic/siem

import { HOSTS_PAGE } from '../../lib/urls';
import { loginAndWaitForPage } from '../../lib/util/helpers';

describe('Fields Browser', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are some badass tests 😎

@@ -34,7 +29,7 @@ describe('timeline data providers', () => {
dragFromAllHostsToTimeline();

cy.get(TIMELINE_DROPPED_DATA_PROVIDERS, {
timeout: TIMELINE_RENDER_DATA_PROVIDERS_TIMEOUT,
timeout: DEFAULT_TIMEOUT + 10 * 1000,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran it a few times but still got: Timed out retrying: Expected to find element: '[data-test-subj="all-hosts"] [data-react-beautiful-dnd-drag-handle]', but never found it. :(

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm doing some ad-hoc testing of another PR and i'm hitting timeouts in the app itself. It looks like siem-dev is not healthy ATM.

Copy link
Contributor

@stephmilovic stephmilovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once the CI is figured out and my little bitty spec question answered

@elastic elastic deleted a comment from elasticmachine Aug 8, 2019
@elastic elastic deleted a comment from elasticmachine Aug 8, 2019
@andrew-goldstein andrew-goldstein force-pushed the fields-browser-tests branch 2 times, most recently from 32b341d to da3b19c Compare August 8, 2019 21:31
@elastic elastic deleted a comment from elasticmachine Aug 8, 2019
@elastic elastic deleted a comment from elasticmachine Aug 8, 2019
@elastic elastic deleted a comment from elasticmachine Aug 8, 2019
@andrew-goldstein andrew-goldstein force-pushed the fields-browser-tests branch 2 times, most recently from 42471a1 to 486e2d8 Compare August 10, 2019 01:47
@jgowdyelastic jgowdyelastic requested review from a team as code owners August 12, 2019 17:36
@jgowdyelastic jgowdyelastic requested a review from a team August 12, 2019 17:36
@elastic elastic deleted a comment from elasticmachine Aug 12, 2019
@cchaos
Copy link
Contributor

cchaos commented Aug 12, 2019

You may want to look into the color contrast ratios of your pin and comment icons. From what I'm seeing in the screenshot it doesn't meet AA standards.

Screen Shot 2019-08-12 at 14 06 28 PM

Instead, you might consider using the $euiColorDarkShade (during dark mode) for the fill:

Screen Shot 2019-08-12 at 14 09 10 PM

Also, we don't tend to rotate icons to indicate state. We already have a pinFilled icon to indicate currently pinned items. It's also a lot easier to scan and notice the contrast between the two icons. An easy way to test is to just blur your eyes a bit and see which one is easier to distinguish the differences.

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@andrew-goldstein
Copy link
Contributor Author

Thanks for the feedback re: contrast and icon rotation @cchaos !

As noted in our out-of-band conversation, the pinFilled icon was added to EUI after the SIEM app adopted pin rotation as the means to differentiate between the pinned and un-pinned states.

Since pinFilled is now available and preferred over rotation to indicate the pinned state, the app has been updated to use the pinFilled icon instead of rotation.

Also, the previous implementation of pinning used EuiIcon with the subdued color, but as noted, the version of the subdued color that ships with EuiIcon does not meet the AA color contrast standard in dark mode. We changed the implementation (as suggested) from EuiIcon to EuiButtonIcon, because the subdued color that ships with EuiButtonIcon has the correct contrast in dark mode:

Before:

unpinned-before

After (unpinned):
after-unpinned

After (pinned):
after-pinned

## Features

-  A `Toggle column` checkbox for adding / removing fields in the timeline offers
an alternative to drag and drop in an expanded event:

![toggle-field](https://user-images.githubusercontent.com/4459398/62572146-4d94c280-b850-11e9-8ff9-5feaca3f305d.gif)

The behavior of the `Toggle field` checkbox in the expanded event is the same
as the checkboxes that appear in the timeline _Fields Browser_.

## Fixes

- Pinned timeline events now use a "filled" styling when pinned instead of rotation, and unpinned events have a higher contrast in dark mode:

Before:

![unpinned-before](https://user-images.githubusercontent.com/4459398/62572175-5b4a4800-b850-11e9-9d4f-317d5abad002.png)

After (unpinned):
![after-unpinned](https://user-images.githubusercontent.com/4459398/62908225-28aebc80-bd34-11e9-9237-8a98c88a6da1.png)

After (pinned):
![after-pinned](https://user-images.githubusercontent.com/4459398/62908230-306e6100-bd34-11e9-9830-45d725bac5b1.png)

- Fixed an issue where the `Copy to Clipboard` icon rendered incorrectly (too
small, and not the right color):

Before:

![copy-to-clipboard-before](https://user-images.githubusercontent.com/4459398/62572224-7452f900-b850-11e9-962c-eef0b96202ff.png)

After:

![copy-to-clipboard-after](https://user-images.githubusercontent.com/4459398/62572240-7f0d8e00-b850-11e9-8b89-7c14bf92e705.png)

- Fixed an issue where a previously invisible control became visible and
occupied additonal row space in the timeline:

Before:

![row-height-before](https://user-images.githubusercontent.com/4459398/62572265-8e8cd700-b850-11e9-9c36-1d74ba0ba6f5.png)

After:

![row-height-after](https://user-images.githubusercontent.com/4459398/62572285-977da880-b850-11e9-8448-7085bbac8da4.png)

- Fixed an issue where the timeline Fields Browser Overlaps Column Headers

Before:

![field-browser-position-before](https://user-images.githubusercontent.com/4459398/62572309-a5332e00-b850-11e9-9b6f-38700d2f48a0.png)

After:

![field-browser-position-after](https://user-images.githubusercontent.com/4459398/62572335-b845fe00-b850-11e9-84f8-4b273f7ef163.png)

## Code Coverage and Cypress Tests

- This PR increases the `jest` unit test coverage for the _Fields Browser_, and
introduces `Cypress` tests for it
- Added new `Cypress` test helpers, configuration, and test refactorings to
support running smoke tests against remote Kibana instances, including
[Elastic Cloud](https://www.elastic.co/cloud)
- Refactored some `Cypress` tests to speed them up and reduce flakyness

elastic/siem-team#380
elastic/siem-team#431
elastic/siem-team#432
elastic/siem-team#433
elastic/siem-team#434
elastic/siem-team#435
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@elastic elastic deleted a comment from elasticmachine Aug 13, 2019
@andrew-goldstein andrew-goldstein merged commit e731aef into elastic:master Aug 13, 2019
@andrew-goldstein andrew-goldstein deleted the fields-browser-tests branch August 13, 2019 04:51
jloleysens added a commit to jloleysens/kibana that referenced this pull request Aug 13, 2019
…p-metrics-selectall

* 'master' of github.com:elastic/kibana: (27 commits)
  [ML] Data Frames: Analytics job creation. (elastic#43102)
  [Vis Default editor] Fix issue with Rollup (elastic#42430)
  [Vis: Default editor] EUIficate Markdown tab (elastic#42677)
  [New Platform Migration Phase I]: update dateHisogramInterval & parseEsInterval imports (elastic#42917)
  [Infra UI] Add AWS metrics to node detail page (elastic#42153)
  update apm index pattern (elastic#43106)
  [SIEM] Toggle Column / Code Coverage and Cypress (elastic#42766)
  skip failing test (elastic#43163)
  [code] Add option to turn the go dependency download on/off. (elastic#43096)
  disable visual regression jobs
  Removed dead code (elastic#42774)
  fixes csv export of saved searches that have _source field (elastic#43123)
  Export missing Context types (elastic#43051)
  Update dependency supports-color to v7 (elastic#43064)
  switch to icon type string instead of node (elastic#43111)
  [Maps] Enable borders for icon symbols (elastic#43066)
  [ftr] enable visualRegression jobs (elastic#42989)
  [ML] Converting single to multi metric job (elastic#42532)
  fix(NA): dont clean dll module if it is a package json file (elastic#42904)
  [Logs UI] Add link from the sample web logs to the Logs UI (elastic#42635)
  ...
andrew-goldstein added a commit to andrew-goldstein/kibana that referenced this pull request Aug 13, 2019
## Features

-  A `Toggle column` checkbox for adding / removing fields in the timeline offers
an alternative to drag and drop in an expanded event:

![toggle-field](https://user-images.githubusercontent.com/4459398/62572146-4d94c280-b850-11e9-8ff9-5feaca3f305d.gif)

The behavior of the `Toggle field` checkbox in the expanded event is the same
as the checkboxes that appear in the timeline _Fields Browser_.

## Fixes

- Pinned timeline events now use a "filled" styling when pinned instead of rotation, and unpinned events have a higher contrast in dark mode:

Before:

![unpinned-before](https://user-images.githubusercontent.com/4459398/62572175-5b4a4800-b850-11e9-9d4f-317d5abad002.png)

After (unpinned):
![after-unpinned](https://user-images.githubusercontent.com/4459398/62908225-28aebc80-bd34-11e9-9237-8a98c88a6da1.png)

After (pinned):
![after-pinned](https://user-images.githubusercontent.com/4459398/62908230-306e6100-bd34-11e9-9830-45d725bac5b1.png)

- Fixed an issue where the `Copy to Clipboard` icon rendered incorrectly (too
small, and not the right color):

Before:

![copy-to-clipboard-before](https://user-images.githubusercontent.com/4459398/62572224-7452f900-b850-11e9-962c-eef0b96202ff.png)

After:

![copy-to-clipboard-after](https://user-images.githubusercontent.com/4459398/62572240-7f0d8e00-b850-11e9-8b89-7c14bf92e705.png)

- Fixed an issue where a previously invisible control became visible and
occupied additonal row space in the timeline:

Before:

![row-height-before](https://user-images.githubusercontent.com/4459398/62572265-8e8cd700-b850-11e9-9c36-1d74ba0ba6f5.png)

After:

![row-height-after](https://user-images.githubusercontent.com/4459398/62572285-977da880-b850-11e9-8448-7085bbac8da4.png)

- Fixed an issue where the timeline Fields Browser Overlaps Column Headers

Before:

![field-browser-position-before](https://user-images.githubusercontent.com/4459398/62572309-a5332e00-b850-11e9-9b6f-38700d2f48a0.png)

After:

![field-browser-position-after](https://user-images.githubusercontent.com/4459398/62572335-b845fe00-b850-11e9-84f8-4b273f7ef163.png)

## Code Coverage and Cypress Tests

- This PR increases the `jest` unit test coverage for the _Fields Browser_, and
introduces `Cypress` tests for it
- Added new `Cypress` test helpers, configuration, and test refactorings to
support running smoke tests against remote Kibana instances, including
[Elastic Cloud](https://www.elastic.co/cloud)
- Refactored some `Cypress` tests to speed them up and reduce flakyness

elastic/siem-team#380
elastic/siem-team#431
elastic/siem-team#432
elastic/siem-team#433
elastic/siem-team#434
elastic/siem-team#435
andrew-goldstein added a commit that referenced this pull request Aug 13, 2019
## Features

-  A `Toggle column` checkbox for adding / removing fields in the timeline offers
an alternative to drag and drop in an expanded event:

![toggle-field](https://user-images.githubusercontent.com/4459398/62572146-4d94c280-b850-11e9-8ff9-5feaca3f305d.gif)

The behavior of the `Toggle field` checkbox in the expanded event is the same
as the checkboxes that appear in the timeline _Fields Browser_.

## Fixes

- Pinned timeline events now use a "filled" styling when pinned instead of rotation, and unpinned events have a higher contrast in dark mode:

Before:

![unpinned-before](https://user-images.githubusercontent.com/4459398/62572175-5b4a4800-b850-11e9-9d4f-317d5abad002.png)

After (unpinned):
![after-unpinned](https://user-images.githubusercontent.com/4459398/62908225-28aebc80-bd34-11e9-9237-8a98c88a6da1.png)

After (pinned):
![after-pinned](https://user-images.githubusercontent.com/4459398/62908230-306e6100-bd34-11e9-9830-45d725bac5b1.png)

- Fixed an issue where the `Copy to Clipboard` icon rendered incorrectly (too
small, and not the right color):

Before:

![copy-to-clipboard-before](https://user-images.githubusercontent.com/4459398/62572224-7452f900-b850-11e9-962c-eef0b96202ff.png)

After:

![copy-to-clipboard-after](https://user-images.githubusercontent.com/4459398/62572240-7f0d8e00-b850-11e9-8b89-7c14bf92e705.png)

- Fixed an issue where a previously invisible control became visible and
occupied additonal row space in the timeline:

Before:

![row-height-before](https://user-images.githubusercontent.com/4459398/62572265-8e8cd700-b850-11e9-9c36-1d74ba0ba6f5.png)

After:

![row-height-after](https://user-images.githubusercontent.com/4459398/62572285-977da880-b850-11e9-8448-7085bbac8da4.png)

- Fixed an issue where the timeline Fields Browser Overlaps Column Headers

Before:

![field-browser-position-before](https://user-images.githubusercontent.com/4459398/62572309-a5332e00-b850-11e9-9b6f-38700d2f48a0.png)

After:

![field-browser-position-after](https://user-images.githubusercontent.com/4459398/62572335-b845fe00-b850-11e9-84f8-4b273f7ef163.png)

## Code Coverage and Cypress Tests

- This PR increases the `jest` unit test coverage for the _Fields Browser_, and
introduces `Cypress` tests for it
- Added new `Cypress` test helpers, configuration, and test refactorings to
support running smoke tests against remote Kibana instances, including
[Elastic Cloud](https://www.elastic.co/cloud)
- Refactored some `Cypress` tests to speed them up and reduce flakyness

elastic/siem-team#380
elastic/siem-team#431
elastic/siem-team#432
elastic/siem-team#433
elastic/siem-team#434
elastic/siem-team#435
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

andrew-goldstein added a commit to andrew-goldstein/kibana that referenced this pull request Aug 14, 2019
…ne / Reporting

This PR contains enhancements to the `Cypress` end-to-end (e2e) testing support in the SIEM app, as discussed with @spalger

Note: this PR depends on test refactorings in elastic#42766

## New: Run `Cypress` Tests Against Elastic Cloud

Cypress tests may now be run against a remote Elastic Cloud instance
(override `baseUrl`), interactively or via the command line.

Credentials are specified via `kibna.dev.yml` or environment variables.

## New: Run `Cypress` Tests On the Command Line

Run tests on the command line (override `baseUrl` and pass credentials via
the `CYPRESS_ELASTICSEARCH_USERNAME` and `CYPRESS_ELASTICSEARCH_PASSWORD`
environment variables), via command line.

## New: Reporting

Reports are configured to include:

- An HTML report, suitable for e-mail notifications
- Screenshots
- Junit reports (for integration with Kibana CI)
- Videos (optional)

Reports generated when `Cypress` tests are run on the command line are output
to the `target` directory, which follows specific conventions used by the
Kibana CI process:

* Any directory under `target` that begins with `kibana-`, e.g. `kibana-siem` will be uploaded as an artifact to a bucket
* Junit reports are picked up from the `target/junit` directory

The artifacts generated by running `Cypress` tests on the command line conform to the above conventions.

### HTML Reports

An HTML report (e.g. for email notifications) is output to:

```
target/kibana-siem/cypress/results/output.html
```

### Screenshots

Screenshots of failed tests are output to:

```
target/kibana-siem/cypress/screenshots
```

### `junit` Reports

The Kibana CI process reports `junit` test results from the `target/junit` directory.

Cypress `junit` reports are generated in `target/kibana-siem/cypress/results`
and copied to the `target/junit` directory.

### Videos (optional)

Videos are disabled by default, but can optionally be enabled by setting the
`CYPRESS_video=true` environment variable:

```
CYPRESS_video=true yarn cypress:run
```

Videos are (optionally) output to:

```
target/kibana-siem/cypress/videos
```

# Updated `README`

The full details and instructions for running Cypress tests are in the
`siem/cypress/README.md`, which was updated to reflect the changes
in this in this PR, but in a nutshell, you may run tests interactively with
the following (new) environment variables:

```sh
cd x-pack/legacy/plugins/siem
CYPRESS_baseUrl=http://localhost:5601 CYPRESS_ELASTICSEARCH_USERNAME=elastic CYPRESS_ELASTICSEARCH_PASSWORD=<password> yarn cypress:open
```

Running the command line version of the tests, which will output
the reports described above, is a similar experence:

```sh
cd x-pack/legacy/plugins/siem
CYPRESS_baseUrl=http://localhost:5601 CYPRESS_ELASTICSEARCH_USERNAME=elastic CYPRESS_ELASTICSEARCH_PASSWORD=<password> yarn cypress:run
```

elastic/siem-team#435
elastic/siem-team#437
andrew-goldstein added a commit that referenced this pull request Aug 14, 2019
… / Reporting (#42804)

# [SIEM] Run Cypress Tests Against Elastic Cloud & Cypress Command Line / Reporting (#42804)

This PR contains enhancements to the `Cypress` end-to-end (e2e) testing support in the SIEM app, as discussed with @spalger

Note: this PR depends on test refactorings in #42766

## New: Run `Cypress` Tests Against Elastic Cloud

Cypress tests may now be run against a remote Elastic Cloud instance
(override `baseUrl`), interactively or via the command line.

Credentials are specified via `kibna.dev.yml` or environment variables.

## New: Run `Cypress` Tests On the Command Line

Run tests on the command line (override `baseUrl` and pass credentials via
the `CYPRESS_ELASTICSEARCH_USERNAME` and `CYPRESS_ELASTICSEARCH_PASSWORD`
environment variables), via command line.

## New: Reporting

Reports are configured to include:

- An HTML report, suitable for e-mail notifications
- Screenshots
- Junit reports (for integration with Kibana CI)
- Videos (optional)

Reports generated when `Cypress` tests are run on the command line are output
to the `target` directory, which follows specific conventions used by the
Kibana CI process:

* Any directory under `target` that begins with `kibana-`, e.g. `kibana-siem` will be uploaded as an artifact to a bucket
* Junit reports are picked up from the `target/junit` directory

The artifacts generated by running `Cypress` tests on the command line conform to the above conventions.

### HTML Reports

An HTML report (e.g. for email notifications) is output to:

```
target/kibana-siem/cypress/results/output.html
```

### Screenshots

Screenshots of failed tests are output to:

```
target/kibana-siem/cypress/screenshots
```

### `junit` Reports

The Kibana CI process reports `junit` test results from the `target/junit` directory.

Cypress `junit` reports are generated in `target/kibana-siem/cypress/results`
and copied to the `target/junit` directory.

### Videos (optional)

Videos are disabled by default, but can optionally be enabled by setting the
`CYPRESS_video=true` environment variable:

```
CYPRESS_video=true yarn cypress:run
```

Videos are (optionally) output to:

```
target/kibana-siem/cypress/videos
```

# Updated `README`

The full details and instructions for running Cypress tests are in the
`siem/cypress/README.md`, which was updated to reflect the changes
in this in this PR, but in a nutshell, you may run tests interactively with
the following (new) environment variables:

```sh
cd x-pack/legacy/plugins/siem
CYPRESS_baseUrl=http://localhost:5601 CYPRESS_ELASTICSEARCH_USERNAME=elastic CYPRESS_ELASTICSEARCH_PASSWORD=<password> yarn cypress:open
```

Running the command line version of the tests, which will output
the reports described above, is a similar experence:

```sh
cd x-pack/legacy/plugins/siem
CYPRESS_baseUrl=http://localhost:5601 CYPRESS_ELASTICSEARCH_USERNAME=elastic CYPRESS_ELASTICSEARCH_PASSWORD=<password> yarn cypress:run
```

elastic/siem-team#435
elastic/siem-team#437
andrew-goldstein added a commit to andrew-goldstein/kibana that referenced this pull request Aug 14, 2019
… / Reporting (elastic#42804)

This PR contains enhancements to the `Cypress` end-to-end (e2e) testing support in the SIEM app, as discussed with @spalger

Note: this PR depends on test refactorings in elastic#42766

Cypress tests may now be run against a remote Elastic Cloud instance
(override `baseUrl`), interactively or via the command line.

Credentials are specified via `kibna.dev.yml` or environment variables.

Run tests on the command line (override `baseUrl` and pass credentials via
the `CYPRESS_ELASTICSEARCH_USERNAME` and `CYPRESS_ELASTICSEARCH_PASSWORD`
environment variables), via command line.

Reports are configured to include:

- An HTML report, suitable for e-mail notifications
- Screenshots
- Junit reports (for integration with Kibana CI)
- Videos (optional)

Reports generated when `Cypress` tests are run on the command line are output
to the `target` directory, which follows specific conventions used by the
Kibana CI process:

* Any directory under `target` that begins with `kibana-`, e.g. `kibana-siem` will be uploaded as an artifact to a bucket
* Junit reports are picked up from the `target/junit` directory

The artifacts generated by running `Cypress` tests on the command line conform to the above conventions.

An HTML report (e.g. for email notifications) is output to:

```
target/kibana-siem/cypress/results/output.html
```

Screenshots of failed tests are output to:

```
target/kibana-siem/cypress/screenshots
```

The Kibana CI process reports `junit` test results from the `target/junit` directory.

Cypress `junit` reports are generated in `target/kibana-siem/cypress/results`
and copied to the `target/junit` directory.

Videos are disabled by default, but can optionally be enabled by setting the
`CYPRESS_video=true` environment variable:

```
CYPRESS_video=true yarn cypress:run
```

Videos are (optionally) output to:

```
target/kibana-siem/cypress/videos
```

The full details and instructions for running Cypress tests are in the
`siem/cypress/README.md`, which was updated to reflect the changes
in this in this PR, but in a nutshell, you may run tests interactively with
the following (new) environment variables:

```sh
cd x-pack/legacy/plugins/siem
CYPRESS_baseUrl=http://localhost:5601 CYPRESS_ELASTICSEARCH_USERNAME=elastic CYPRESS_ELASTICSEARCH_PASSWORD=<password> yarn cypress:open
```

Running the command line version of the tests, which will output
the reports described above, is a similar experence:

```sh
cd x-pack/legacy/plugins/siem
CYPRESS_baseUrl=http://localhost:5601 CYPRESS_ELASTICSEARCH_USERNAME=elastic CYPRESS_ELASTICSEARCH_PASSWORD=<password> yarn cypress:run
```

elastic/siem-team#435
elastic/siem-team#437

- regen yarn.lock
andrew-goldstein added a commit that referenced this pull request Aug 14, 2019
… / Reporting (#42804) (#43315)

This PR contains enhancements to the `Cypress` end-to-end (e2e) testing support in the SIEM app, as discussed with @spalger

Note: this PR depends on test refactorings in #42766

Cypress tests may now be run against a remote Elastic Cloud instance
(override `baseUrl`), interactively or via the command line.

Credentials are specified via `kibna.dev.yml` or environment variables.

Run tests on the command line (override `baseUrl` and pass credentials via
the `CYPRESS_ELASTICSEARCH_USERNAME` and `CYPRESS_ELASTICSEARCH_PASSWORD`
environment variables), via command line.

Reports are configured to include:

- An HTML report, suitable for e-mail notifications
- Screenshots
- Junit reports (for integration with Kibana CI)
- Videos (optional)

Reports generated when `Cypress` tests are run on the command line are output
to the `target` directory, which follows specific conventions used by the
Kibana CI process:

* Any directory under `target` that begins with `kibana-`, e.g. `kibana-siem` will be uploaded as an artifact to a bucket
* Junit reports are picked up from the `target/junit` directory

The artifacts generated by running `Cypress` tests on the command line conform to the above conventions.

An HTML report (e.g. for email notifications) is output to:

```
target/kibana-siem/cypress/results/output.html
```

Screenshots of failed tests are output to:

```
target/kibana-siem/cypress/screenshots
```

The Kibana CI process reports `junit` test results from the `target/junit` directory.

Cypress `junit` reports are generated in `target/kibana-siem/cypress/results`
and copied to the `target/junit` directory.

Videos are disabled by default, but can optionally be enabled by setting the
`CYPRESS_video=true` environment variable:

```
CYPRESS_video=true yarn cypress:run
```

Videos are (optionally) output to:

```
target/kibana-siem/cypress/videos
```

The full details and instructions for running Cypress tests are in the
`siem/cypress/README.md`, which was updated to reflect the changes
in this in this PR, but in a nutshell, you may run tests interactively with
the following (new) environment variables:

```sh
cd x-pack/legacy/plugins/siem
CYPRESS_baseUrl=http://localhost:5601 CYPRESS_ELASTICSEARCH_USERNAME=elastic CYPRESS_ELASTICSEARCH_PASSWORD=<password> yarn cypress:open
```

Running the command line version of the tests, which will output
the reports described above, is a similar experence:

```sh
cd x-pack/legacy/plugins/siem
CYPRESS_baseUrl=http://localhost:5601 CYPRESS_ELASTICSEARCH_USERNAME=elastic CYPRESS_ELASTICSEARCH_PASSWORD=<password> yarn cypress:run
```

elastic/siem-team#435
elastic/siem-team#437

- regen yarn.lock
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants