Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FTR] Add test suite metrics tracking/output #62515

Merged
merged 18 commits into from
Apr 20, 2020

Conversation

brianseeders
Copy link
Contributor

Uses FTR lifecycle hooks to track the success and start/end time of each individual test suite run, and then outputs that data to a JSON file.

A separate PR will upload/download these metrics to create optimized functional test plans based on historical data.

@brianseeders brianseeders added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.7.0 v7.8.0 v7.6.3 labels Apr 3, 2020
@brianseeders brianseeders added the Team:Operations Team label for Operations Team label Apr 9, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-operations (Team:Operations)

@brianseeders brianseeders requested a review from a team April 9, 2020 19:34
@brianseeders brianseeders marked this pull request as ready for review April 9, 2020 19:35
@brianseeders
Copy link
Contributor Author

@elasticmachine merge upstream

@brianseeders
Copy link
Contributor Author

@elasticmachine merge upstream

@brianseeders
Copy link
Contributor Author

@elasticmachine merge upstream

@LeeDr LeeDr mentioned this pull request Apr 17, 2020
29 tasks
Copy link
Contributor

@spalger spalger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@brianseeders
Copy link
Contributor Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@brianseeders brianseeders merged commit 3ce3da8 into elastic:master Apr 20, 2020
@brianseeders brianseeders deleted the add-ftr-metrics branch April 20, 2020 20:15
brianseeders added a commit to brianseeders/kibana that referenced this pull request Apr 20, 2020
brianseeders added a commit to brianseeders/kibana that referenced this pull request Apr 20, 2020
brianseeders added a commit to brianseeders/kibana that referenced this pull request Apr 20, 2020
gmmorris added a commit to gmmorris/kibana that referenced this pull request Apr 21, 2020
* master: (30 commits)
  Migrate vis_type_table to kibana/new platform  (elastic#63105)
  Enable include/exclude in Terms agg for numeric fields (elastic#59425)
  follow conventions for saved object definitions (elastic#63571)
  [Docs]Adds saved object key setting to load balancing kib instances (elastic#63935)
  kbn/config-schema: Consider maybe properties as optional keys in ObjectType (elastic#63838)
  Fix page layouts, clean up unused code (elastic#63992)
  [SIEM] Adds recursive exact key checks for validation and formatter
  [Maps] Migrate remaining maps client files to NP (except routi… (elastic#63859)
  [Maps] Do not fetch geo_shape from docvalues (elastic#63997)
  Vega doc changes (elastic#63889)
  [Metrics UI] Reorganize file layout for Metrics UI (elastic#60049)
  Add sub-menus to Resolver node (for 75% zoom) (elastic#63476)
  [FTR] Add test suite metrics tracking/output (elastic#62515)
  [ML] Fixing single metric viewer page padding (elastic#63839)
  [Discover] Allow user to generate a report after saving a modified saved search (elastic#63623)
  [Reporting] Config flag to escape formula CSV values (elastic#63645)
  [Metrics UI] Remove remaining field filtering (elastic#63398)
  [Maps] fix date labels (elastic#63909)
  [TSVB] Use default Kibana palette for split series (elastic#62241)
  Ingest overview page (elastic#63214)
  ...
jloleysens added a commit that referenced this pull request Apr 21, 2020
…t-node-pipelines

* 'master' of github.com:elastic/kibana: (32 commits)
  Move authz lib out of snapshot restore (#63947)
  Migrate vis_type_table to kibana/new platform  (#63105)
  Enable include/exclude in Terms agg for numeric fields (#59425)
  follow conventions for saved object definitions (#63571)
  [Docs]Adds saved object key setting to load balancing kib instances (#63935)
  kbn/config-schema: Consider maybe properties as optional keys in ObjectType (#63838)
  Fix page layouts, clean up unused code (#63992)
  [SIEM] Adds recursive exact key checks for validation and formatter
  [Maps] Migrate remaining maps client files to NP (except routi… (#63859)
  [Maps] Do not fetch geo_shape from docvalues (#63997)
  Vega doc changes (#63889)
  [Metrics UI] Reorganize file layout for Metrics UI (#60049)
  Add sub-menus to Resolver node (for 75% zoom) (#63476)
  [FTR] Add test suite metrics tracking/output (#62515)
  [ML] Fixing single metric viewer page padding (#63839)
  [Discover] Allow user to generate a report after saving a modified saved search (#63623)
  [Reporting] Config flag to escape formula CSV values (#63645)
  [Metrics UI] Remove remaining field filtering (#63398)
  [Maps] fix date labels (#63909)
  [TSVB] Use default Kibana palette for split series (#62241)
  ...
jloleysens added a commit to jloleysens/kibana that referenced this pull request Apr 21, 2020
…bana into ingest-node-pipelines/privileges

* 'feature/ingest-node-pipelines' of github.com:elastic/kibana: (34 commits)
  Move authz lib out of snapshot restore (elastic#63947)
  Migrate vis_type_table to kibana/new platform  (elastic#63105)
  Enable include/exclude in Terms agg for numeric fields (elastic#59425)
  follow conventions for saved object definitions (elastic#63571)
  [Docs]Adds saved object key setting to load balancing kib instances (elastic#63935)
  kbn/config-schema: Consider maybe properties as optional keys in ObjectType (elastic#63838)
  Fix page layouts, clean up unused code (elastic#63992)
  [SIEM] Adds recursive exact key checks for validation and formatter
  [Maps] Migrate remaining maps client files to NP (except routi… (elastic#63859)
  [Maps] Do not fetch geo_shape from docvalues (elastic#63997)
  Vega doc changes (elastic#63889)
  [Metrics UI] Reorganize file layout for Metrics UI (elastic#60049)
  Add sub-menus to Resolver node (for 75% zoom) (elastic#63476)
  [FTR] Add test suite metrics tracking/output (elastic#62515)
  [Ingest pipelines] Delete pipeline (elastic#63635)
  [ML] Fixing single metric viewer page padding (elastic#63839)
  [Discover] Allow user to generate a report after saving a modified saved search (elastic#63623)
  [Reporting] Config flag to escape formula CSV values (elastic#63645)
  [Metrics UI] Remove remaining field filtering (elastic#63398)
  [Maps] fix date labels (elastic#63909)
  ...

# Conflicts:
#	x-pack/legacy/plugins/uptime/public/components/monitor/ml/index.ts
#	x-pack/legacy/plugins/uptime/public/components/overview/index.ts
#	x-pack/plugins/ingest_pipelines/public/application/index.tsx
#	x-pack/plugins/ingest_pipelines/server/routes/api/index.ts
#	x-pack/plugins/ingest_pipelines/server/routes/index.ts
@kibanamachine
Copy link
Contributor

Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync.

@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Apr 21, 2020
jloleysens added a commit to jloleysens/kibana that referenced this pull request Apr 22, 2020
…bana into pipeline-editor-part-mvp-2

* 'feature/ingest-node-pipelines' of github.com:elastic/kibana: (34 commits)
  [Ingest Node Pipelines] Clone Pipeline (elastic#64049)
  Move authz lib out of snapshot restore (elastic#63947)
  Migrate vis_type_table to kibana/new platform  (elastic#63105)
  Enable include/exclude in Terms agg for numeric fields (elastic#59425)
  follow conventions for saved object definitions (elastic#63571)
  [Docs]Adds saved object key setting to load balancing kib instances (elastic#63935)
  kbn/config-schema: Consider maybe properties as optional keys in ObjectType (elastic#63838)
  Fix page layouts, clean up unused code (elastic#63992)
  [SIEM] Adds recursive exact key checks for validation and formatter
  [Maps] Migrate remaining maps client files to NP (except routi… (elastic#63859)
  [Maps] Do not fetch geo_shape from docvalues (elastic#63997)
  Vega doc changes (elastic#63889)
  [Metrics UI] Reorganize file layout for Metrics UI (elastic#60049)
  Add sub-menus to Resolver node (for 75% zoom) (elastic#63476)
  [FTR] Add test suite metrics tracking/output (elastic#62515)
  [Ingest pipelines] Delete pipeline (elastic#63635)
  [ML] Fixing single metric viewer page padding (elastic#63839)
  [Discover] Allow user to generate a report after saving a modified saved search (elastic#63623)
  [Reporting] Config flag to escape formula CSV values (elastic#63645)
  [Metrics UI] Remove remaining field filtering (elastic#63398)
  ...
@kibanamachine
Copy link
Contributor

Looks like this PR has backport PRs but they still haven't been merged. Please merge them ASAP to keep the branches relatively in sync.

brianseeders added a commit that referenced this pull request Apr 23, 2020
@kibanamachine kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Apr 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes Team:Operations Team label for Operations Team v7.6.3 v7.7.0 v7.8.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants