Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

beat/kibana module - add {module}.elasticsearch.cluster.id property #29577

Merged
merged 16 commits into from
Dec 26, 2021

Conversation

klacabane
Copy link
Contributor

@klacabane klacabane commented Dec 22, 2021

Summary

This change adds the beat.elasticsearch.cluster.id property to both state and stats beat's metricsets, which is required for the SM UI to retrieve beats that outputs their events to a selected cluster.
It also reverts this change as we should not be writing to other root module properties from a given module. This requires alias changes in elasticsearch to be functional.

The cluster ID was already available in the state metricset so it was just a matter of adding that property to the right path. For the stats metricset I reused the 7.16 xpack logic.

Testing

  • start one metricbeat with elasticsearch-xpack module enabled
  • start another metricbeat that monitors the first one
  • verify relevant beats information appear in the SM UI

@klacabane klacabane added Team:Integrations Label for the Integrations team Feature:Stack Monitoring v8.0.0 backport-v8.0.0 Automated backport with mergify v8.1.0 labels Dec 22, 2021
@klacabane klacabane self-assigned this Dec 22, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/stack-monitoring (Stack monitoring)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Dec 22, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented Dec 22, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-12-24T13:45:03.966+0000

  • Duration: 106 min 25 sec

  • Commit: b790e82

Test stats 🧪

Test Results
Failed 0
Passed 9928
Skipped 2550
Total 12478

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

"running": c.Int("running"),
"starts": c.Int("starts"),
"stops": c.Int("stops"),
"config": c.Dict("config", s.Schema{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll need to add those fields in _meta/fields.yml

@klacabane klacabane changed the title beat module - add cluster.id property beat/kibana module - add {module}.elasticsearch.cluster.id property Dec 23, 2021
@matschaffer
Copy link
Contributor

It's possible I've got something mis-wired here, but not getting any beats monitoring data yet

Screen Shot 2021-12-24 at 13 20 07

@klacabane
Copy link
Contributor Author

@matschaffer did your ES build include these mappings change ? What type of beat did you monitor and did it output to ES ?

Copy link
Contributor

@sayden sayden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Please, add a Changelog entry. Thanks!

@klacabane klacabane merged commit 90dd169 into elastic:master Dec 26, 2021
mergify bot pushed a commit that referenced this pull request Dec 26, 2021
…29577)

* add elasticsearch.cluster.id property to beats state

* add elasticsearch.cluster.id property to beats stats

* fix test

* lint

* add missing reloads property

* dont break earlier assumptions

* remove trailing comma

* update beats.stats fields

* add updated fields doc

* beat: update cluster.id path

* kibana: update cluster.id path

* mage fmt update

* fix fields path

* add missing map initialization

* add changelog entry

* fix pr number

(cherry picked from commit 90dd169)
klacabane added a commit that referenced this pull request Dec 27, 2021
…h.cluster.id property (#29608)

* beat/kibana module - add {module}.elasticsearch.cluster.id property (#29577)

* add elasticsearch.cluster.id property to beats state

* add elasticsearch.cluster.id property to beats stats

* fix test

* lint

* add missing reloads property

* dont break earlier assumptions

* remove trailing comma

* update beats.stats fields

* add updated fields doc

* beat: update cluster.id path

* kibana: update cluster.id path

* mage fmt update

* fix fields path

* add missing map initialization

* add changelog entry

* fix pr number

(cherry picked from commit 90dd169)

* remove non-backported changeset entries

Co-authored-by: Kevin Lacabane <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v8.0.0 Automated backport with mergify Feature:Stack Monitoring Team:Integrations Label for the Integrations team v8.0.0 v8.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants