Support kube_state_metrics v2.0.0 #27552

MichaelKatsoulis · 2021-08-23T12:59:59Z

What does this PR do?

This PR enables Metricbeat to support kube-state-metrics version 2.0.0 for kubernetes.

In v2.0.0 some breaking changes where introduced.

Following fields where removed and replaced by kube_pod_container_resource_requests and kube_pod_container_resource_limits.
- kube_pod_container_resource_requests_cpu_cores
- kube_pod_container_resource_limits_cpu_cores
- kube_pod_container_resource_requests_memory_bytes
- kube_pod_container_resource_limits_memory_bytes
Following fields where removed and replaced by kube_node_status_capacity and kube_node_status_allocatable.
- kube_node_status_capacity_pods
- kube_node_status_capacity_cpu_cores
- kube_node_status_capacity_memory_bytes
- kube_node_status_allocatable_pods
- kube_node_status_allocatable_cpu_cores
- kube_node_status_allocatable_memory_bytes

Metricbeat was updated to support both new and old format of those fields.

Why is it important?

After version 2.0.0 of kube-state-metrics some fields where deprecated and replaced by others.
Without this PR fields

cpu.request.cores
cpu.limit.cores
memory.limit.bytes
memory.request.bytes
pod.capacity.total
cpu.capacity.cores
memory.capacity.bytes
pod.allocatable.total
cpu.allocatable.cores
memory.allocatable.bytes

where missing when kube-state-metrics v2.0.0 and higher was present in the Kubernetes
cluster.

Details can be found here.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Run metricbeat as daemonset in a Kubernetes cluster where kube-state-metrics v2.0.0 or higher is deployed and watch those fields getting populated in Elasticsearch.

Related issues

Closes Metricbeat to support kube-state-metrics version 2.0.0 #21520

Screenshots

…esource_limits

elasticmachine · 2021-08-23T13:45:17Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-08-26T09:24:57.956+0000
Duration: 83 min 51 sec
Commit: 27fec71

Test stats 🧪

Test	Results
Failed	0
Passed	9125
Skipped	2412
Total	11537

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	9125
Skipped	2412
Total	11537

elasticmachine · 2021-08-23T15:04:25Z

Pinging @elastic/integrations (Team:Integrations)

ChrsMark

Addition looks good to me! I left a minor code related question/suggestion.

I'm not sure if we miss anything else in order to support v2.0.0 but I guess we have already done our research and we concluded to the fields that we are handling here, right?
Also I would suggest to list in the PR's description the fields that we handle in this PR for v2.0.0. so as to have clear/quick access to them in the future if we have questions about v2 support.

ChrsMark · 2021-08-25T08:29:01Z

metricbeat/module/kubernetes/state_container/state_container.go

@@ -130,18 +142,29 @@ func (m *MetricSet) Fetch(reporter mb.ReporterV2) error {
 	}

 	m.enricher.Enrich(events)
-
+	//m.Logger().Infof("Events are %+v", events)


ChrsMark · 2021-08-25T08:32:26Z

metricbeat/module/kubernetes/state_container/state_container.go

-		if limit, ok := event["cpu.limit.cores"]; ok {
-			if limitCores, ok := limit.(float64); ok {
-				event["cpu.limit.nanocores"] = limitCores * nanocores
+		if cpuFields, ok := event["cpu"]; ok {


Why not to convert the event to common.MapStr and then access the cpu.limit.cores field directly, so as to avoid all these ifs? With the methods that MapStr provides you should be ok.

Good catch. event is already a common.MapStr

ChrsMark · 2021-08-25T08:33:28Z

metricbeat/module/kubernetes/state_container/state_container.go

-			"kube_pod_container_info":                           p.InfoMetric(),
+			"kube_pod_info":           p.InfoMetric(),
+			"kube_pod_container_info": p.InfoMetric(),
+			"kube_pod_container_resource_requests": p.Metric("", p.OpFilterMap(


Great that you were able to re-use our existing OpFilter to handle this!

ChrsMark · 2021-08-25T08:38:25Z

metricbeat/module/kubernetes/state_container/_meta/test/ksm.v1.3.0.expected

@@ -21,7 +21,8 @@
 		"MetricSetFields": {
 			"cpu": {
 				"request": {
-					"cores": 0.15


Also I think that we need to cover ksm.v2.0.0 in our tests, right?

MichaelKatsoulis · 2021-08-26T09:03:52Z

@ChrsMark It is updated and ready for review

ChrsMark

lgtm, left some nits

ChrsMark · 2021-08-26T09:14:12Z

metricbeat/module/kubernetes/state_container/state_container.go

@@ -130,21 +142,19 @@ func (m *MetricSet) Fetch(reporter mb.ReporterV2) error {
 	}

 	m.enricher.Enrich(events)
-


nit: consider not removing lines if there is no specific reason

ChrsMark · 2021-08-26T09:16:22Z

metricbeat/module/kubernetes/state_container/state_container.go

 			}
 		}
-


nit: same here, do not remove lines if it's not intentional

* Support kube_pod_container_resource_requests and kube_pod_container_resource_limits * Add kube_node_status_capacity and kube_node_status_allocatable support * Add tests for ksm v2 for all state data streams (cherry picked from commit 1596d64)

* Support kube_pod_container_resource_requests and kube_pod_container_resource_limits * Add kube_node_status_capacity and kube_node_status_allocatable support * Add tests for ksm v2 for all state data streams (cherry picked from commit 1596d64) Co-authored-by: Michael Katsoulis <[email protected]>

* master: Forward port 7.14.1 changelog to master (elastic#27687) Addressing multiple dashboard issues: deps loading once, field conversion, etc. (elastic#27669) Remove adaptive queue sizes from agent's spec files (elastic#27653) Osquerybeat: Improve testability and unit test coverage (elastic#27591) Osquerybeat: lockdown flagsfile, prevent global defaults (elastic#27611) Import the references of dashboard assets using the Saved Objects API (elastic#27647) Fix bug with override path in cgroups (elastic#27620) Allow Kibana client to authorize with Elasticsearch API key (elastic#27540) Filebeat auditd: Fix Top Exec Commands dashboard visualization (elastic#27638) [elastic-agent] Fix docker tar.gz generation for complete image (elastic#27621) Follow up changes in dashboards in mage check && fix minor issue (elastic#27553) [Heartbeat] Fix bug where `enabled: false` is ignored. (elastic#27615) Support kube_state_metrics v2.0.0 (elastic#27552)

* Support kube_pod_container_resource_requests and kube_pod_container_resource_limits * Add kube_node_status_capacity and kube_node_status_allocatable support * Add tests for ksm v2 for all state data streams (cherry picked from commit 1596d64)

* Support kube_state_metrics v2.0.0 (#27552) * Support kube_pod_container_resource_requests and kube_pod_container_resource_limits * Add kube_node_status_capacity and kube_node_status_allocatable support * Add tests for ksm v2 for all state data streams (cherry picked from commit 1596d64) * Update CHANGELOG.next.asciidoc

* Support kube_pod_container_resource_requests and kube_pod_container_resource_limits * Add kube_node_status_capacity and kube_node_status_allocatable support * Add tests for ksm v2 for all state data streams

Support kube_pod_container_resource_requests and kube_pod_container_r…

84fc11d

…esource_limits

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 23, 2021

MichaelKatsoulis marked this pull request as draft August 23, 2021 13:00

Add kube_node_status_capacity and kube_node_status_allocatable support

bd17339

MichaelKatsoulis added the Team:Integrations Label for the Integrations team label Aug 23, 2021

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 23, 2021

MichaelKatsoulis added 7.16-candidate enhancement labels Aug 23, 2021

MichaelKatsoulis requested a review from ChrsMark August 23, 2021 14:41

MichaelKatsoulis changed the title ~~Support kube_pod_container_resource_requests and kube_pod_container_resource_limits~~ Support kube_state_metrics v2.0.0 Aug 23, 2021

Update changelog

5521546

MichaelKatsoulis marked this pull request as ready for review August 23, 2021 15:04

Add cpu requests and limits nanocores in test files

14a9c79

MichaelKatsoulis mentioned this pull request Aug 24, 2021

Prepare requested resources visualizations for Kubernetes Observability elastic/integrations#1014

Closed

ChrsMark reviewed Aug 25, 2021

View reviewed changes

MichaelKatsoulis added 3 commits August 25, 2021 14:33

Review update

daba6cd

Add tests fro kube-state-metrics v2.0.0

eda32e1

Add tests for ksm v2 for all state data streams

827ae31

ChrsMark approved these changes Aug 26, 2021

View reviewed changes

ChrsMark reviewed Aug 26, 2021

View reviewed changes

metricbeat/module/kubernetes/state_container/state_container.go

}

}

Copy link

Member

ChrsMark Aug 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: same here, do not remove lines if it's not intentional

Add deleted lines

27fec71

MichaelKatsoulis added the backport-v7.16.0 Automated backport with mergify label Aug 26, 2021

MichaelKatsoulis merged commit 1596d64 into elastic:master Aug 26, 2021

mergify bot mentioned this pull request Aug 26, 2021

[7.x](backport #27552) Support kube_state_metrics v2.0.0 #27606

Merged

MichaelKatsoulis mentioned this pull request Sep 22, 2021

Cherry-pick #27552 to 7.15: Support kube_state_metrics v2.0.0 #28049

Merged

6 tasks

MichaelKatsoulis added the v7.15.0 label Sep 22, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support kube_state_metrics v2.0.0 #27552

Support kube_state_metrics v2.0.0 #27552

MichaelKatsoulis commented Aug 23, 2021 •

edited

Loading

elasticmachine commented Aug 23, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

Trends 🧪

Test stats 🧪

elasticmachine commented Aug 23, 2021

ChrsMark left a comment

ChrsMark Aug 25, 2021

ChrsMark Aug 25, 2021

MichaelKatsoulis Aug 25, 2021

ChrsMark Aug 25, 2021

ChrsMark Aug 25, 2021

MichaelKatsoulis commented Aug 26, 2021

ChrsMark left a comment

ChrsMark Aug 26, 2021

ChrsMark Aug 26, 2021

		@@ -130,21 +142,19 @@ func (m *MetricSet) Fetch(reporter mb.ReporterV2) error {
		}

		m.enricher.Enrich(events)

Support kube_state_metrics v2.0.0 #27552

Support kube_state_metrics v2.0.0 #27552

Conversation

MichaelKatsoulis commented Aug 23, 2021 • edited Loading

What does this PR do?

Why is it important?

Checklist

How to test this PR locally

Related issues

Screenshots

elasticmachine commented Aug 23, 2021 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

elasticmachine commented Aug 23, 2021

ChrsMark left a comment

Choose a reason for hiding this comment

ChrsMark Aug 25, 2021

Choose a reason for hiding this comment

ChrsMark Aug 25, 2021

Choose a reason for hiding this comment

MichaelKatsoulis Aug 25, 2021

Choose a reason for hiding this comment

ChrsMark Aug 25, 2021

Choose a reason for hiding this comment

ChrsMark Aug 25, 2021

Choose a reason for hiding this comment

MichaelKatsoulis commented Aug 26, 2021

ChrsMark left a comment

Choose a reason for hiding this comment

ChrsMark Aug 26, 2021

Choose a reason for hiding this comment

ChrsMark Aug 26, 2021

Choose a reason for hiding this comment

MichaelKatsoulis commented Aug 23, 2021 •

edited

Loading

elasticmachine commented Aug 23, 2021 •

edited by jenkins-beats-ci bot

Loading