Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ftr] add ability to get included/excluded test counts #25760

Merged
merged 4 commits into from
Nov 16, 2018

Conversation

spalger
Copy link
Contributor

@spalger spalger commented Nov 15, 2018

Summary

This pr adds a --test-stats flag to the functional test runner that will allow extracting information from a specific config/tags combo about how many tests will run and how many are excluded. This is necessary as part of #22359 so we can effectively ignore specific configs (avoiding starting es and kibana) when there aren't any tests that will be run.

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

- [ ] This was checked for cross-browser compatibility, including a check against IE11
- [ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
- [ ] Documentation was added for features that require explanation or tutorials
- [ ] Unit or functional tests were updated or added to match the most common scenarios
- [ ] This was checked for keyboard-only and screenreader accessibility

For maintainers

- [ ] This was checked for breaking API changes and was labeled appropriately
- [ ] This includes a feature addition or change that requires a release note and was labeled appropriately

@spalger spalger added review Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v7.0.0 v6.6.0 labels Nov 15, 2018
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-platform

@spalger spalger force-pushed the implement/ftr-test-stats branch from 404f358 to c694c52 Compare November 15, 2018 23:26
@spalger spalger force-pushed the implement/ftr-test-stats branch from c694c52 to 315bf42 Compare November 15, 2018 23:26
@spalger spalger requested a review from joshdover November 15, 2018 23:42
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Copy link
Contributor

@joshdover joshdover left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

src/functional_test_runner/functional_test_runner.js Outdated Show resolved Hide resolved
@elastic elastic deleted a comment from elasticmachine Nov 16, 2018
@spalger spalger merged commit f7832a5 into elastic:master Nov 16, 2018
spalger pushed a commit to spalger/kibana that referenced this pull request Nov 16, 2018
## Summary

This pr adds a `--test-stats` flag to the functional test runner that will allow extracting information from a specific config/tags combo about how many tests will run and how many are excluded. This is necessary as part of elastic#22359 so we can effectively ignore specific configs (avoiding starting es and kibana) when there aren't any tests that will be run.

### Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~
~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~
~~- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~
~~- [ ] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios~~
~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~

### For maintainers

~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~
~~- [ ] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~
spalger pushed a commit that referenced this pull request Nov 17, 2018
## Summary

This pr adds a `--test-stats` flag to the functional test runner that will allow extracting information from a specific config/tags combo about how many tests will run and how many are excluded. This is necessary as part of #22359 so we can effectively ignore specific configs (avoiding starting es and kibana) when there aren't any tests that will be run.

### Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~
~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~
~~- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~
~~- [ ] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios~~
~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~

### For maintainers

~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~
~~- [ ] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~
@spalger
Copy link
Contributor Author

spalger commented Nov 17, 2018

6.x/6.6: cc2a051

@spalger spalger deleted the implement/ftr-test-stats branch November 17, 2018 01:21
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
review Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v6.6.0 v7.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants