-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Complete doc on running functional tests #13035
Conversation
@@ -8,17 +8,59 @@ We use functional tests to make sure the Kibana UI works as expected. It replace | |||
|
|||
The `FunctionalTestRunner` is very bare bones and gets most of its functionality from its config file, located at {blob}test/functional/config.js[test/functional/config.js]. If you’re writing a plugin you will have your own config file. See <<development-plugin-functional-tests>> for more info. | |||
|
|||
Execute the `FunctionalTestRunner`'s script with node.js to run the tests with Kibana's default configuration: | |||
|
|||
The following will start Kibana, Elasticsearch and the chromedriver for you. To run the functional UI tests use the following commands |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm interpreting these instructions as having to run 3 commands, npm run test:ui
, then npm run test:ui:server
, and then finally node scripts/functional_test_runner
. Is this interpretation correct?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ycombinator I think that how you interpreted the instructions is logical, but that is not correct.
I think this section should be given three sub-sections:
- There are three ways to run the tests depending on your goals:
-
Easiest option, much slower when running the tests multiple times, runs elasticsearch/kibana locally:
npm run test:ui
- does everything in a single command
- tears down everything after the tests run
- exit code reports success/failure of the tests
-
Optimized for development, tests can be re-run much faster, runs es/kibana locally:
- two commands, run in separate terminals, separates the components that are long-running and slow from those that are ephemeral and fast.
npm run test:ui:server
- starts elasticsearch and the kibana server
- slow to start
- can be reused for multiple executions of the tests
- automatically restarts the kibana server when relevant changes are detected
node scripts/functional_test_runner
- runs the tests against kibana/elasticsearch that were started
npm run test:ui:server
- exit code reports success/failure of the tests
- runs the tests against kibana/elasticsearch that were started
- two commands, run in separate terminals, separates the components that are long-running and slow from those that are ephemeral and fast.
-
Runs test tests against instances of Elasticsearch/Kibana started some other way (like elastic cloud, or an instance you are managing some other way)
- just executes the functional tests
- url, credentials, etc. for Elasticsearch and Kibana are specified via environment variables
export TEST_KIBANA_PROTOCOL=https export TEST_KIBANA_HOSTNAME=my-kibana-instance.internal.net export TEST_KIBANA_PORT=443 export TEST_KIBANA_USER=kibana export TEST_KIBANA_PASS=password export TEST_ES_PROTOCOL=http export TEST_ES_HOSTNAME=my-es-cluster.internal.net export TEST_ES_PORT=9200 export TEST_ES_USER=elastic export TEST_ES_PASS=password node scripts/functional_test_runner
-
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perfect, I suspected something wasn't quite right. Thanks for clarifying and the suggestions @spalger!
Cool. So @spalger I will update the docs to reflect your comment. Essentially, your comment will become the docs. |
Take another look, please? @spalger @ycombinator |
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Love the descriptions of the options! LGTM.
@archanid Github can't parse asciidoc. For doc changes, you should always compile the docs locally to make sure they do what you want and can actually compile, since a broken docs page in Kibana will prevent docs from publishing across all projects. https://github.com/elastic/docs is the docs repo, and its readme shows instructions for getting it running. Note that when building docs for Kibana, you'll want to use the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, though please make sure things look right by building locally before merging
@spalger or @epixa is there documentation for how to compile the docs locally? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am getting a warning when building
asciidoc: WARNING: development-functional-tests.asciidoc: line 53: section title out of sequence: expected level 3, got level 4
@tylersmalley try again? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
* Complete doc on running functional tests * Update development-functional-tests.asciidoc * Fix section title
* Complete doc on running functional tests * Update development-functional-tests.asciidoc * Fix section title
* Complete doc on running functional tests * Update development-functional-tests.asciidoc * Fix section title
* Complete doc on running functional tests * Update development-functional-tests.asciidoc * Fix section title
* Complete doc on running functional tests * Update development-functional-tests.asciidoc * Fix section title
Addresses #11797