Skip to content

Commit

Permalink
Add plugin testing notes, clean up test section formatting
Browse files Browse the repository at this point in the history
  • Loading branch information
Rashid Khan committed Mar 1, 2016
1 parent 705a7b4 commit 0f07dac
Showing 1 changed file with 50 additions and 59 deletions.
109 changes: 50 additions & 59 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -111,99 +111,90 @@ Before running the tests you will need to install the projects dependencies as d

Once that is complete just run:

```sh
```
sh
npm run test && npm run build
```

#### Testing and debugging tests
#### Debugging unit tests

The standard `npm run test` task runs several sub tasks and can take several minutes to complete, making debugging failures pretty painful. In order to ease the pain specialized tasks provide alternate methods for running the tests.

<dl>
<dt><code>npm run test:quick</code></dt>
<dd>Runs both server and browser tests, but skips linting</dd>

<dt><code>npm run test:server</code> or <code>npm run test:browser</code></dt>
<dd>Runs the tests for just the server or browser</dd>

<dt><code>npm run test:dev</code></dt>
<dd>
Initializes an environment for debugging the browser tests. Includes an dedicated instance of the kibana server for building the test bundle, and a karma server. When running this task the build is optimized for the first time and then a karma-owned instance of the browser is opened. Click the "debug" button to open a new tab that executes the unit tests.
<br>
<img src="http://i.imgur.com/DwHxgfq.png">
</dd>

<dt><code>npm run mocha [test file or dir]</code> or <code>npm run mocha:debug [test file or dir]</code></dt>
<dd>
Run a one off test with the local project version of mocha, babel compilation, and optional debugging. Great
for development and fixing individual tests.
</dd>

<dt><code>npm run test:dev -- --kbnServer.testsBundle.pluginId=some_special_plugin --kbnServer.plugin-path=../some_special_plugin</code></dt>
<dd>
Run the tests for just your particular plugin
</dd>
</dl>

Distributable packages can be found in `target/` after the build completes.
`npm run test:quick`
Runs both server and browser tests, but skips linting

#### Building OS packages
`npm run test:server`
Run only the server tests

Packages are built using fpm, pleaserun, dpkg, and rpm. fpm and pleaserun can be installed using gem. Package building has only been tested on Linux and is not supported on any other platform.
```sh
gem install pleaserun
apt-get install ruby-dev
gem install fpm
npm run build:ospackages
```
`npm run test:browser`
Run only the browser tests

To specify a package to build you can add `rpm` or `deb` as an argument.
```sh
npm run build:ospackages -- --rpm
```
`npm run test:dev`
Initializes an environment for debugging the browser tests. Includes an dedicated instance of the kibana server for building the test bundle, and a karma server. When running this task the build is optimized for the first time and then a karma-owned instance of the browser is opened. Click the "debug" button to open a new tab that executes the unit tests.
![Browser test debugging](http://i.imgur.com/DwHxgfq.png)

### Functional UI Testing
`npm run mocha [test file or dir]` or `npm run mocha:debug [test file or dir]`
Run a one off test with the local project version of mocha, babel compilation, and optional debugging. Great
for development and fixing individual tests.

#### Handy references
#### Unit testing plugins
This should work super if you're using the [Kibana plugin generator](https://github.com/elastic/generator-kibana-plugin). If you're not using the generator, well, you're on your own. We suggest you look at how the generator works.

- https://theintern.github.io/
- https://theintern.github.io/leadfoot/Element.html
`npm run test:dev -- --kbnServer.testsBundle.pluginId=some_special_plugin --kbnServer.plugin-path=../some_special_plugin`
Run the tests for just your particular plugin. Assuming you plugin lives outside of the `installedPlugins directory`, which it should.

#### Running tests using npm task:
#### Running browser automation tests:

*The Selenium server that is started currently only runs the tests in Firefox*

To run the functional UI tests use the following commands

<dl>
The following will start Kibana, Elasticsearch and Selenium for you. To run the functional UI tests use the following commands

<dt><code>npm run test:ui</code></dt>
<dd>Run the functional UI tests one time and exit. This is used by the CI systems and is great for quickly checking that things pass. It is essentially a combination of the next two tasks.</dd>
`npm run test:ui`
Run the functional UI tests one time and exit. This is used by the CI systems and is great for quickly checking that things pass. It is essentially a combination of the next two tasks.

<dt><code>npm run test:ui:server</code></dt>
<dd>Start the server required for the <code>test:ui:runner</code> tasks. Once the server is started <code>test:ui:runner</code> can be run multiple times without waiting for the server to start.</dd>
`npm run test:ui:server`
Start the server required for the `test:ui:runner` tasks. Once the server is started `test:ui:runner` can be run multiple times without waiting for the server to start.

<dt><code>npm run test:ui:runner</code></dt>
<dd>Execute the front-end selenium tests. This requires the server started by the <code>test:ui:server</code> task.</dd>
`npm run test:ui:runner`
Execute the front-end selenium tests. This requires the server started by the `test:ui:server` task.

</dl>

#### Running tests locally with your existing (and already running) ElasticSearch, Kibana, and Selenium Server:
##### If you already have ElasticSearch, Kibana, and Selenium Server running:

Set your es and kibana ports in `test/intern.js` to 9220 and 5620, respectively. You can configure your Selenium server to run the tests on Chrome,IE, or other browsers here.

Once you've got the services running, execute the following:

```sh
```
sh
npm run test:ui:runner
```

#### General notes:
#### Browser automation notes:

- Using Page Objects pattern (https://theintern.github.io/intern/#writing-functional-test)
- At least the initial tests for the Settings, Discover, and Visualize tabs all depend on a very specific set of logstash-type data (generated with makelogs). Since that is a static set of data, all the Discover and Visualize tests use a specific Absolute time range. This guarantees the same results each run.
- These tests have been developed and tested with Chrome and Firefox browser. In theory, they should work on all browsers (that's the benefit of Intern using Leadfoot).
- These tests should also work with an external testing service like https://saucelabs.com/ or https://www.browserstack.com/ but that has not been tested.
- https://theintern.github.io/
- https://theintern.github.io/leadfoot/Element.html

#### Building OS packages

Packages are built using fpm, pleaserun, dpkg, and rpm. fpm and pleaserun can be installed using gem. Package building has only been tested on Linux and is not supported on any other platform.
```sh
gem install pleaserun
apt-get install ruby-dev
gem install fpm
npm run build:ospackages
```

To specify a package to build you can add `rpm` or `deb` as an argument.
```sh
npm run build:ospackages -- --rpm
```

Distributable packages can be found in `target/` after the build completes.

## Submitting a pull request

Expand Down

0 comments on commit 0f07dac

Please sign in to comment.