Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix kibana metricsets property paths #122034

Merged
merged 10 commits into from
Dec 31, 2021

Conversation

klacabane
Copy link
Contributor

@klacabane klacabane commented Dec 27, 2021

Summary

The metricbeat path for kibana queries currently expects an inner kibana.stats.kibana object to contain a summary of the instance similar to the legacy schema, but this object does not exist. Instead the properties making up this summary are spread in the kibana.stats object. This change updates codepaths relying on this non-existing object.

Also updated the _mb kibana archives (with this script) used by the apis and functional tests to 1. align with the actual metricbeat schema 2. insert documents in the new datastreams.

Testing

  • start metricbeat with kibana module enabled
  • ensure all Stack monitoring kibana views are loading and displaying relevant data

@klacabane klacabane self-assigned this Dec 28, 2021
@klacabane klacabane added auto-backport Deprecated - use backport:version if exact versions are needed backport-v8.0.0 Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services v8.0.0 v8.1.0 labels Dec 28, 2021
@klacabane klacabane marked this pull request as ready for review December 28, 2021 16:24
@klacabane klacabane requested a review from a team as a code owner December 28, 2021 16:24
@elasticmachine
Copy link
Contributor

Pinging @elastic/infra-monitoring-ui (Team:Infra Monitoring UI)

@klacabane klacabane added the release_note:skip Skip the PR/issue when compiling release notes label Dec 28, 2021
@klacabane
Copy link
Contributor Author

@elasticmachine merge upstream

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @klacabane

Copy link
Contributor

@miltonhultgren miltonhultgren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes look good 👍🏼

@@ -23,6 +23,7 @@ export default function ({ getService, getPageObjects }) {
{
from: 'Aug 29, 2017 @ 17:24:14.254',
to: 'Aug 29, 2017 @ 17:25:44.142',
useCreate: true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of scope: This is set since we're writing to a data stream right? I wonder if perhaps it would be nicer if our setup methods used that domain language instead of "useCreate" which is a detail of how es_archiver decides to put data into ES if it's a data stream?

It's very unclear to me from just reading this call site what this parameter does.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, I'll update that as part of the logstash PR

const legacyStats = hit._source.kibana_stats;
const mbStats = hit._source.kibana?.stats;

const kibana: Kibana = {
kibana: hit._source.kibana?.kibana ?? legacyStats?.kibana,
kibana: buildKibanaInfo(hit),
concurrent_connections:
mbStats?.concurrent_connections ?? legacyStats?.concurrent_connections,
process: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe out of scope: But for the code below, I wonder if we can write some helper function like getProperty(propertyName, hit), which would hide the legacyDats, mbStats, and all the optional chaining and ?? operators?
I wonder if that would make it a bit more readable?

Internally I'd imagine we give a definite name to each property we use, and two functions that know how to find that property from a hit, then the function itself just grabs these to selectors, and chains them together (i the first one is null, use the second one) to find the values.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be a good improvement for sure. similar conditionals/fallbacks are spread everywhere in the codebase so I'd rather handle this as part of a separate refactoring effort :)

@kibanamachine
Copy link
Contributor

💚 Backport successful

Status Branch Result
8.0

This backport PR will be merged automatically after passing CI.

kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Dec 31, 2021
* fix kibana metricsets property paths

* return inner kibana object

* fix functional test

* rename

* adapt tests to datastream

* rebuild archived data

* extract buildKibanaInfo for reuse

* add license

* lint

Co-authored-by: Kibana Machine <[email protected]>
kibanamachine added a commit that referenced this pull request Jan 3, 2022
* fix kibana metricsets property paths

* return inner kibana object

* fix functional test

* rename

* adapt tests to datastream

* rebuild archived data

* extract buildKibanaInfo for reuse

* add license

* lint

Co-authored-by: Kibana Machine <[email protected]>

Co-authored-by: Kevin Lacabane <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Deprecated - use backport:version if exact versions are needed release_note:skip Skip the PR/issue when compiling release notes Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services v8.0.0 v8.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants