Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete doc on running functional tests #13035

Merged
merged 3 commits into from
Aug 29, 2017
Merged

Complete doc on running functional tests #13035

merged 3 commits into from
Aug 29, 2017

Conversation

rhoboat
Copy link

@rhoboat rhoboat commented Jul 21, 2017

Addresses #11797

@rhoboat rhoboat requested review from ycombinator and spalger July 21, 2017 16:36
@rhoboat rhoboat self-assigned this Jul 21, 2017
@rhoboat rhoboat requested a review from epixa July 21, 2017 16:37
@@ -8,17 +8,59 @@ We use functional tests to make sure the Kibana UI works as expected. It replace

The `FunctionalTestRunner` is very bare bones and gets most of its functionality from its config file, located at {blob}test/functional/config.js[test/functional/config.js]. If you’re writing a plugin you will have your own config file. See <<development-plugin-functional-tests>> for more info.

Execute the `FunctionalTestRunner`'s script with node.js to run the tests with Kibana's default configuration:

The following will start Kibana, Elasticsearch and the chromedriver for you. To run the functional UI tests use the following commands
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm interpreting these instructions as having to run 3 commands, npm run test:ui, then npm run test:ui:server, and then finally node scripts/functional_test_runner. Is this interpretation correct?

Copy link
Contributor

@spalger spalger Jul 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ycombinator I think that how you interpreted the instructions is logical, but that is not correct.

I think this section should be given three sub-sections:

  • There are three ways to run the tests depending on your goals:
    1. Easiest option, much slower when running the tests multiple times, runs elasticsearch/kibana locally:

      • npm run test:ui
        • does everything in a single command
        • tears down everything after the tests run
        • exit code reports success/failure of the tests
    2. Optimized for development, tests can be re-run much faster, runs es/kibana locally:

      • two commands, run in separate terminals, separates the components that are long-running and slow from those that are ephemeral and fast.
        • npm run test:ui:server
          • starts elasticsearch and the kibana server
          • slow to start
          • can be reused for multiple executions of the tests
          • automatically restarts the kibana server when relevant changes are detected
        • node scripts/functional_test_runner
          • runs the tests against kibana/elasticsearch that were started npm run test:ui:server
          • exit code reports success/failure of the tests
    3. Runs test tests against instances of Elasticsearch/Kibana started some other way (like elastic cloud, or an instance you are managing some other way)

      • just executes the functional tests
      • url, credentials, etc. for Elasticsearch and Kibana are specified via environment variables
      export TEST_KIBANA_PROTOCOL=https
      export TEST_KIBANA_HOSTNAME=my-kibana-instance.internal.net
      export TEST_KIBANA_PORT=443
      export TEST_KIBANA_USER=kibana
      export TEST_KIBANA_PASS=password
      
      export TEST_ES_PROTOCOL=http
      export TEST_ES_HOSTNAME=my-es-cluster.internal.net
      export TEST_ES_PORT=9200
      export TEST_ES_USER=elastic
      export TEST_ES_PASS=password
      node scripts/functional_test_runner

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, I suspected something wasn't quite right. Thanks for clarifying and the suggestions @spalger!

@rhoboat
Copy link
Author

rhoboat commented Jul 25, 2017

Cool. So @spalger I will update the docs to reflect your comment. Essentially, your comment will become the docs.

@rhoboat rhoboat added Team:Operations Team label for Operations Team Team:Docs labels Jul 25, 2017
@rhoboat
Copy link
Author

rhoboat commented Jul 25, 2017

Take another look, please? @spalger @ycombinator

@rhoboat
Copy link
Author

rhoboat commented Jul 25, 2017

{blob}test/functional/config.js[test/functional/config.js] isn't rendering as I'd like. Maybe because github doesn't know how to properly render asciidoc. I'd like to publish and see how it renders there, tweaking if needed at that point.

Copy link
Contributor

@ycombinator ycombinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the descriptions of the options! LGTM.

@epixa
Copy link
Contributor

epixa commented Jul 25, 2017

@archanid Github can't parse asciidoc. For doc changes, you should always compile the docs locally to make sure they do what you want and can actually compile, since a broken docs page in Kibana will prevent docs from publishing across all projects.

https://github.com/elastic/docs is the docs repo, and its readme shows instructions for getting it running.

Note that when building docs for Kibana, you'll want to use the --chunk=1 option as well, which isn't documented.

Copy link
Contributor

@spalger spalger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, though please make sure things look right by building locally before merging

@rhoboat
Copy link
Author

rhoboat commented Aug 7, 2017

@spalger or @epixa is there documentation for how to compile the docs locally?
Edit: nevermind, I think this is it https://github.com/elastic/docs#for-a-local-repo

@epixa
Copy link
Contributor

epixa commented Aug 7, 2017

@archanid Those are the correct docs. I have a PR up to improve the doc building experience for Kibana, and I expect that to get merged in the next couple of days: #13164

Copy link
Contributor

@tylersmalley tylersmalley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am getting a warning when building

asciidoc: WARNING: development-functional-tests.asciidoc: line 53: section title out of sequence: expected level 3, got level 4

@rhoboat rhoboat closed this Aug 22, 2017
@rhoboat rhoboat deleted the func-test-doc branch August 22, 2017 20:58
@rhoboat rhoboat restored the func-test-doc branch August 22, 2017 21:24
@rhoboat rhoboat reopened this Aug 22, 2017
@rhoboat
Copy link
Author

rhoboat commented Aug 22, 2017

@tylersmalley try again?

Copy link
Contributor

@tylersmalley tylersmalley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rhoboat rhoboat merged commit a019594 into elastic:master Aug 29, 2017
@rhoboat rhoboat deleted the func-test-doc branch August 29, 2017 21:49
rhoboat pushed a commit that referenced this pull request Aug 29, 2017
* Complete doc on running functional tests

* Update development-functional-tests.asciidoc

* Fix section title
rhoboat pushed a commit that referenced this pull request Aug 29, 2017
* Complete doc on running functional tests

* Update development-functional-tests.asciidoc

* Fix section title
rhoboat pushed a commit that referenced this pull request Aug 29, 2017
* Complete doc on running functional tests

* Update development-functional-tests.asciidoc

* Fix section title
rhoboat pushed a commit that referenced this pull request Aug 29, 2017
* Complete doc on running functional tests

* Update development-functional-tests.asciidoc

* Fix section title
rhoboat pushed a commit that referenced this pull request Aug 29, 2017
* Complete doc on running functional tests

* Update development-functional-tests.asciidoc

* Fix section title
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants