Add missing cluster metadata to multiple k8s metricsests #32979

tetianakravchenko · 2022-09-05T16:17:42Z

What does this PR do?

Add orchestrator.cluster.name and orchestrator.cluster.ip fields to the datasets: proxy, system, apiserver, scheduler and controllermanager.

Why is it important?

orchestrator.cluster.name is used for kubernetes dashboards

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Related [K8s] cluster metadata not added on all metrics #30425

Use cases

Screenshots

Logs

Signed-off-by: Tetiana Kravchenko <[email protected]>

mergify · 2022-09-05T16:18:19Z

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @tetianakravchenko? 🙏.
For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

elasticmachine · 2022-09-05T16:49:43Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-09-07T15:01:25.032+0000
Duration: 50 min 46 sec

Test stats 🧪

Test	Results
Failed	0
Passed	3725
Skipped	873
Total	4598

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Signed-off-by: Tetiana Kravchenko <[email protected]>

tetianakravchenko · 2022-09-07T07:48:33Z

/package

ChrsMark

lgtm, left one minor comment.

ChrsMark · 2022-09-07T14:06:38Z

metricbeat/module/kubernetes/util/kubernetes.go

@@ -574,3 +574,24 @@ func GetClusterECSMeta(cfg *conf.C, client k8sclient.Interface, logger *logp.Log
 	}
 	return ecsClusterMeta, nil
 }
+
+// AddClusterECSMeta adds ECS orchestrator fields
+func AddClusterECSMeta(base mb.BaseMetricSet) mapstr.M {


Can this be also used at https://github.com/elastic/beats/blob/main/metricbeat/module/kubernetes/event/event.go#L102

@ChrsMark I think no, for event is used different default config - https://github.com/elastic/beats/blob/main/metricbeat/module/kubernetes/event/event.go#L65, comparing to other metricsets.

I've replaced adding ECS orchestratop for volume - afa7fc6, as it is the same as for proxy, system, apiserver, scheduler and controllermanager

gizas · 2022-09-07T14:14:55Z

metricbeat/module/kubernetes/apiserver/apiserver.go

-			"apiserver_audit_requests_rejected_total": prometheus.Metric("audit.rejected.count"),
-			"rest_client_requests_total":              prometheus.Metric("client.request.count"),
-		},
+var mapping = &prometheus.MetricsMapping{


As long as you correctly moved them out just you can consider use the bulk var definition as we have them here: https://github.com/elastic/beats/blob/main/metricbeat/module/kubernetes/state_pod/state_pod.go#L36

Just to be the same.

as we have here just one variable mapping I would keep it as is

…dClusterECSMeta Signed-off-by: Tetiana Kravchenko <[email protected]>

* add missing field Signed-off-by: Tetiana Kravchenko <[email protected]> * update log message Signed-off-by: Tetiana Kravchenko <[email protected]> * fix tests Signed-off-by: Tetiana Kravchenko <[email protected]> * add pr number Signed-off-by: Tetiana Kravchenko <[email protected]> * volume metricset: replace adding ECS orchestrator fields with util.AddClusterECSMeta Signed-off-by: Tetiana Kravchenko <[email protected]> Signed-off-by: Tetiana Kravchenko <[email protected]> (cherry picked from commit 1f5b863) # Conflicts: # metricbeat/module/kubernetes/apiserver/_meta/test/metrics.1.14.expected # metricbeat/module/kubernetes/apiserver/apiserver.go # metricbeat/module/kubernetes/apiserver/metricset.go # metricbeat/module/kubernetes/controllermanager/_meta/test/metrics.1.20.expected # metricbeat/module/kubernetes/controllermanager/controllermanager.go

* add missing field Signed-off-by: Tetiana Kravchenko <[email protected]> * update log message Signed-off-by: Tetiana Kravchenko <[email protected]> * fix tests Signed-off-by: Tetiana Kravchenko <[email protected]> * add pr number Signed-off-by: Tetiana Kravchenko <[email protected]> * volume metricset: replace adding ECS orchestrator fields with util.AddClusterECSMeta Signed-off-by: Tetiana Kravchenko <[email protected]> Signed-off-by: Tetiana Kravchenko <[email protected]> (cherry picked from commit 1f5b863)

…3020) * add missing field Signed-off-by: Tetiana Kravchenko <[email protected]> * update log message Signed-off-by: Tetiana Kravchenko <[email protected]> * fix tests Signed-off-by: Tetiana Kravchenko <[email protected]> * add pr number Signed-off-by: Tetiana Kravchenko <[email protected]> * volume metricset: replace adding ECS orchestrator fields with util.AddClusterECSMeta Signed-off-by: Tetiana Kravchenko <[email protected]> Signed-off-by: Tetiana Kravchenko <[email protected]> (cherry picked from commit 1f5b863) Co-authored-by: Tetiana Kravchenko <[email protected]>

…etricsests (#33019) * Add missing cluster metadata to multiple k8s metricsests (#32979) * add missing field Signed-off-by: Tetiana Kravchenko <[email protected]> * update log message Signed-off-by: Tetiana Kravchenko <[email protected]> * fix tests Signed-off-by: Tetiana Kravchenko <[email protected]> * add pr number Signed-off-by: Tetiana Kravchenko <[email protected]> * volume metricset: replace adding ECS orchestrator fields with util.AddClusterECSMeta Signed-off-by: Tetiana Kravchenko <[email protected]> Signed-off-by: Tetiana Kravchenko <[email protected]> (cherry picked from commit 1f5b863) # Conflicts: # metricbeat/module/kubernetes/apiserver/_meta/test/metrics.1.14.expected # metricbeat/module/kubernetes/apiserver/apiserver.go # metricbeat/module/kubernetes/apiserver/metricset.go # metricbeat/module/kubernetes/controllermanager/_meta/test/metrics.1.20.expected # metricbeat/module/kubernetes/controllermanager/controllermanager.go * Update CHANGELOG.next.asciidoc * Update metrics.1.14.expected * Update apiserver.go * Update metricset.go * Update controllermanager.go * Update metrics.1.8.expected * Update metrics.1.20.expected * Update metrics.1.14.expected * Update metrics.1.8.expected * Update metrics.1.14.expected * Update metrics.1.14.expected * Delete metrics.1.20.expected * Update metrics.1.8.expected * Update metrics.1.8.expected * Update metrics.1.14.expected * Update metrics.controllermanager.1.14.expected Co-authored-by: Tetiana Kravchenko <[email protected]>

* add missing field Signed-off-by: Tetiana Kravchenko <[email protected]> * update log message Signed-off-by: Tetiana Kravchenko <[email protected]> * fix tests Signed-off-by: Tetiana Kravchenko <[email protected]> * add pr number Signed-off-by: Tetiana Kravchenko <[email protected]> * volume metricset: replace adding ECS orchestrator fields with util.AddClusterECSMeta Signed-off-by: Tetiana Kravchenko <[email protected]> Signed-off-by: Tetiana Kravchenko <[email protected]>

tetianakravchenko added 2 commits September 5, 2022 17:20

add missing field

76cad82

Signed-off-by: Tetiana Kravchenko <[email protected]>

update log message

8dec97a

Signed-off-by: Tetiana Kravchenko <[email protected]>

tetianakravchenko requested a review from a team as a code owner September 5, 2022 16:17

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 5, 2022

mergify bot assigned tetianakravchenko Sep 5, 2022

tetianakravchenko requested a review from a team September 5, 2022 16:23

tetianakravchenko added the Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team label Sep 5, 2022

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Sep 5, 2022

tetianakravchenko marked this pull request as draft September 5, 2022 18:57

tetianakravchenko added 2 commits September 6, 2022 14:58

fix tests

26c6444

Signed-off-by: Tetiana Kravchenko <[email protected]>

add pr number

375a08f

Signed-off-by: Tetiana Kravchenko <[email protected]>

tetianakravchenko marked this pull request as ready for review September 6, 2022 14:00

tetianakravchenko added backport-v8.3.0 Automated backport with mergify backport-v8.4.0 Automated backport with mergify labels Sep 7, 2022

ChrsMark reviewed Sep 7, 2022

View reviewed changes

gizas reviewed Sep 7, 2022

View reviewed changes

gizas approved these changes Sep 7, 2022

View reviewed changes

volume metricset: replace adding ECS orchestrator fields with util.Ad…

afa7fc6

…dClusterECSMeta Signed-off-by: Tetiana Kravchenko <[email protected]>

tetianakravchenko mentioned this pull request Sep 8, 2022

[K8s] cluster metadata not added on all metrics #30425

Closed

tetianakravchenko merged commit 1f5b863 into elastic:main Sep 8, 2022

mergify bot mentioned this pull request Sep 8, 2022

[8.3](backport #32979) Add missing cluster metadata to multiple k8s metricsests #33019

Merged

mergify bot mentioned this pull request Sep 8, 2022

[8.4](backport #32979) Add missing cluster metadata to multiple k8s metricsests #33020

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add missing cluster metadata to multiple k8s metricsests #32979

Add missing cluster metadata to multiple k8s metricsests #32979

tetianakravchenko commented Sep 5, 2022

mergify bot commented Sep 5, 2022

elasticmachine commented Sep 5, 2022 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

tetianakravchenko commented Sep 7, 2022

ChrsMark left a comment

ChrsMark Sep 7, 2022

tetianakravchenko Sep 7, 2022 •

edited

Loading

gizas Sep 7, 2022

tetianakravchenko Sep 7, 2022

Add missing cluster metadata to multiple k8s metricsests #32979

Add missing cluster metadata to multiple k8s metricsests #32979

Conversation

tetianakravchenko commented Sep 5, 2022

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

mergify bot commented Sep 5, 2022

elasticmachine commented Sep 5, 2022 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

tetianakravchenko commented Sep 7, 2022

ChrsMark left a comment

Choose a reason for hiding this comment

ChrsMark Sep 7, 2022

Choose a reason for hiding this comment

tetianakravchenko Sep 7, 2022 • edited Loading

Choose a reason for hiding this comment

gizas Sep 7, 2022

Choose a reason for hiding this comment

tetianakravchenko Sep 7, 2022

Choose a reason for hiding this comment

elasticmachine commented Sep 5, 2022 •

edited by jenkins-beats-ci bot

Loading

tetianakravchenko Sep 7, 2022 •

edited

Loading