Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Kubernetes] Add data_stream.dataset pre filter is dashboards and remove duplicated dashboard tags #4972

Merged
5 changes: 5 additions & 0 deletions packages/kubernetes/changelog.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.30.0"
changes:
- description: Use datas_tream.dataset as pre filters for dashboards and remove tags
type: enhancement
link: https://github.com/elastic/integrations/pull/4972
- version: "1.29.2"
changes:
- description: Fix function for memory node usage
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
"panelsJSON": "{\"adf38acd-ecc8-48b2-b7f3-d6dfd024e46b\":{\"order\":0,\"width\":\"medium\",\"grow\":true,\"type\":\"optionsListControl\",\"explicitInput\":{\"fieldName\":\"orchestrator.cluster.name\",\"title\":\"Cluster Name\",\"id\":\"adf38acd-ecc8-48b2-b7f3-d6dfd024e46b\",\"enhancements\":{}}},\"3e873627-001d-47c7-91a8-995014b0ef90\":{\"order\":1,\"width\":\"medium\",\"grow\":true,\"type\":\"optionsListControl\",\"explicitInput\":{\"fieldName\":\"kubernetes.namespace\",\"title\":\"Namespace Name\",\"id\":\"3e873627-001d-47c7-91a8-995014b0ef90\",\"enhancements\":{}}},\"f4b8cf46-4644-4713-872d-dccc4aeb1e44\":{\"order\":2,\"width\":\"medium\",\"grow\":true,\"type\":\"optionsListControl\",\"explicitInput\":{\"fieldName\":\"kubernetes.cronjob.name\",\"title\":\"CronJob Name\",\"id\":\"f4b8cf46-4644-4713-872d-dccc4aeb1e44\",\"enhancements\":{}}}}"
},
"description": "Metrics about Cronjobs",
"hits": 0,
"kibanaSavedObjectMeta": {
"searchSourceJSON": {
"filter": [],
Expand Down Expand Up @@ -58,7 +57,7 @@
"panelIndex": "85ecbb8b-9606-4c19-a108-385f825ad7aa",
"title": "Kubernetes Dashboards [Metrics Kubernetes]",
"type": "visualization",
"version": "8.4.0-SNAPSHOT"
"version": "8.6.0-SNAPSHOT"
},
{
"embeddableConfig": {
Expand Down Expand Up @@ -87,7 +86,7 @@
"drop_last_bucket": 0,
"filter": {
"language": "kuery",
"query": "event.dataset :\"kubernetes.state_cronjob\" "
"query": "data_stream.dataset :\"kubernetes.state_cronjob\" "
},
"id": "d9403a9e-99ec-469a-8373-8c33af18e310",
"index_pattern_ref_name": "metrics_37b56399-4cec-4d72-9e9e-e87cbe2d581f_0_index_pattern",
Expand Down Expand Up @@ -144,6 +143,7 @@
"type": "metric",
"use_kibana_indexes": true
},
"title": "",
"type": "metrics",
"uiState": {}
}
Expand All @@ -158,31 +158,27 @@
"panelIndex": "37b56399-4cec-4d72-9e9e-e87cbe2d581f",
"title": "Active CronJobs [Metrics Kubernetes]",
"type": "visualization",
"version": "8.4.0-SNAPSHOT"
"version": "8.6.0-SNAPSHOT"
},
{
"embeddableConfig": {
"attributes": {
"references": [
{
"id": "metrics-*",
"name": "indexpattern-datasource-current-indexpattern",
"type": "index-pattern"
},
{
"id": "metrics-*",
"name": "indexpattern-datasource-layer-7711169c-3a7b-4071-98d0-3644aa1dde0b",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there 2 different metrics-* index patterns? See line 173

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In references section there are five with the same id metrics-* referenced. I don't know how so many got created. I will see if just one is ok

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be the "data view as value" entries. If that is the case, I suspect there will be one per visualisations. I think that is one of the limitations @joshdover stumbled over recently.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes exactly. There Is one per visualizations are all needed for the dash to render

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to remove on of the two in this example but then I get errors in the visualization using it like undefined is not an object (evaluating 't.find((({name:t})=>t===wc(e))).id'). So I will just keep as it is

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created this issue for Kibana team elastic/kibana#149105

"type": "index-pattern"
},
{
"id": "metrics-*",
"name": "eae93c72-eb87-4a07-850b-43393ea55bcf",
"name": "204eb33a-97ff-4d38-bee0-d93387164ab1",
"type": "index-pattern"
}
],
"state": {
"adHocDataViews": {},
"datasourceStates": {
"indexpattern": {
"formBased": {
"layers": {
"7711169c-3a7b-4071-98d0-3644aa1dde0b": {
"columnOrder": [
Expand Down Expand Up @@ -283,21 +279,22 @@
"meta": {
"alias": null,
"disabled": false,
"index": "eae93c72-eb87-4a07-850b-43393ea55bcf",
"key": "event.dataset",
"index": "204eb33a-97ff-4d38-bee0-d93387164ab1",
"key": "data_stream.dataset",
"negate": false,
"params": {
"query": "state_cronjob"
"query": "kubernetes.state_cronjob"
},
"type": "phrase"
},
"query": {
"match_phrase": {
"event.dataset": "kubernetes.state_cronjob"
"data_stream.dataset": "kubernetes.state_cronjob"
}
}
}
],
"internalReferences": [],
"query": {
"language": "kuery",
"query": ""
Expand Down Expand Up @@ -327,6 +324,8 @@
"rowHeightLines": 1
}
},
"title": "CronJobs Informations [Metrics Kubernetes]",
Copy link
Contributor

@tetianakravchenko tetianakravchenko Jan 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we remove this from name [Metrics Kubernetes] here as well? similar to this comment in #4948 (comment)

not for this PR, in general

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for spotting those. In the statefulsets dashboard I missed to update one of the visualizations filters.
For the other ones for Container these were some extra filters used in advance settings of some fields in two visualizations. So it would not be a problem as the visualization was already using the right pre filters on top level. But I updated them all now for consistency.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after commit 43482cc Statefulset dashboard still use event

"query": "event.dataset :\"kubernetes.state_statefulset\" "
, is it expected?

"type": "lens",
"visualizationType": "lnsDatatable"
},
"enhancements": {},
Expand All @@ -342,37 +341,33 @@
"panelIndex": "10f9719c-1b46-4561-b8cf-f3cd3ee69c10",
"title": "CronJobs Informations [Metrics Kubernetes]",
"type": "lens",
"version": "8.4.0-SNAPSHOT"
"version": "8.6.0-SNAPSHOT"
}
],
"timeRestore": false,
"title": "[Metrics Kubernetes] Cronjobs",
"version": 1
},
"coreMigrationVersion": "8.4.0",
"coreMigrationVersion": "8.6.0",
"created_at": "2023-01-11T14:16:01.140Z",
"id": "kubernetes-0a672d50-bcb1-11ec-b64f-7dd6e8e82013",
"migrationVersion": {
"dashboard": "8.3.0"
"dashboard": "8.6.0"
},
"references": [
{
"id": "metrics-*",
"name": "37b56399-4cec-4d72-9e9e-e87cbe2d581f:metrics_37b56399-4cec-4d72-9e9e-e87cbe2d581f_0_index_pattern",
"type": "index-pattern"
},
{
"id": "metrics-*",
"name": "10f9719c-1b46-4561-b8cf-f3cd3ee69c10:indexpattern-datasource-current-indexpattern",
"type": "index-pattern"
},
{
"id": "metrics-*",
"name": "10f9719c-1b46-4561-b8cf-f3cd3ee69c10:indexpattern-datasource-layer-7711169c-3a7b-4071-98d0-3644aa1dde0b",
"type": "index-pattern"
},
{
"id": "metrics-*",
"name": "10f9719c-1b46-4561-b8cf-f3cd3ee69c10:eae93c72-eb87-4a07-850b-43393ea55bcf",
"name": "10f9719c-1b46-4561-b8cf-f3cd3ee69c10:204eb33a-97ff-4d38-bee0-d93387164ab1",
"type": "index-pattern"
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
"panelsJSON": "{\"5f6614ff-57c0-400e-8350-47e86ad5c77f\":{\"order\":0,\"width\":\"medium\",\"grow\":true,\"type\":\"optionsListControl\",\"explicitInput\":{\"fieldName\":\"orchestrator.cluster.name\",\"title\":\"Cluster Name\",\"id\":\"5f6614ff-57c0-400e-8350-47e86ad5c77f\",\"enhancements\":{}}},\"6faac538-d3fd-4f77-85cc-3b7171c7144c\":{\"order\":1,\"width\":\"medium\",\"grow\":true,\"type\":\"optionsListControl\",\"explicitInput\":{\"fieldName\":\"kubernetes.namespace\",\"title\":\"Namespace Name\",\"id\":\"6faac538-d3fd-4f77-85cc-3b7171c7144c\",\"enhancements\":{}}},\"82c41492-acf8-4b51-bba9-ec54c99fb1ba\":{\"order\":2,\"width\":\"medium\",\"grow\":true,\"type\":\"optionsListControl\",\"explicitInput\":{\"fieldName\":\"kubernetes.statefulset.name\",\"title\":\"StatefulSet Name\",\"id\":\"82c41492-acf8-4b51-bba9-ec54c99fb1ba\",\"enhancements\":{}}}}"
},
"description": "Metrics about StatefulSets",
"hits": 0,
"kibanaSavedObjectMeta": {
"searchSourceJSON": {
"filter": [],
Expand Down Expand Up @@ -58,7 +57,7 @@
"panelIndex": "f1e8f8c6-d644-4b1d-a7bc-fe631c232a57",
"title": "Kubernetes Dashboards [Metrics Kubernetes]",
"type": "visualization",
"version": "8.4.0-SNAPSHOT"
"version": "8.6.0-SNAPSHOT"
},
{
"embeddableConfig": {
Expand Down Expand Up @@ -87,7 +86,7 @@
"drop_last_bucket": 0,
"filter": {
"language": "kuery",
"query": "event.dataset :\"kubernetes.state_statefulset\" "
"query": "data_stream.dataset :\"kubernetes.state_statefulset\" "
},
"id": "0db5e639-b021-4535-ab29-629ee94ad72a",
"index_pattern_ref_name": "metrics_d8265dae-829d-434f-a826-cc6062edfd3a_0_index_pattern",
Expand Down Expand Up @@ -144,6 +143,7 @@
"type": "metric",
"use_kibana_indexes": true
},
"title": "",
"type": "metrics",
"uiState": {}
}
Expand All @@ -158,7 +158,7 @@
"panelIndex": "d8265dae-829d-434f-a826-cc6062edfd3a",
"title": "StatefulSet Replicas Observed [Metrics Kubernetes]",
"type": "visualization",
"version": "8.4.0-SNAPSHOT"
"version": "8.6.0-SNAPSHOT"
},
{
"embeddableConfig": {
Expand Down Expand Up @@ -187,7 +187,7 @@
"drop_last_bucket": 0,
"filter": {
"language": "kuery",
"query": "event.dataset :\"kubernetes.state_statefulset\" "
"query": "data_stream.dataset :\"kubernetes.state_statefulset\" "
},
"id": "0db5e639-b021-4535-ab29-629ee94ad72a",
"index_pattern_ref_name": "metrics_21039d12-cc17-4089-9d8f-3c62018c8f1c_0_index_pattern",
Expand Down Expand Up @@ -244,6 +244,7 @@
"type": "metric",
"use_kibana_indexes": true
},
"title": "",
"type": "metrics",
"uiState": {}
}
Expand All @@ -258,7 +259,7 @@
"panelIndex": "21039d12-cc17-4089-9d8f-3c62018c8f1c",
"title": "StatefulSet Replicas Desired [Metrics Kubernetes]",
"type": "visualization",
"version": "8.4.0-SNAPSHOT"
"version": "8.6.0-SNAPSHOT"
},
{
"embeddableConfig": {
Expand Down Expand Up @@ -287,7 +288,7 @@
"drop_last_bucket": 0,
"filter": {
"language": "kuery",
"query": "event.dataset :\"kubernetes.state_statefulset\" "
"query": "data_stream.dataset :\"kubernetes.state_statefulset\" "
},
"id": "0db5e639-b021-4535-ab29-629ee94ad72a",
"index_pattern_ref_name": "metrics_dd7df777-8dae-4374-96ee-461be82fbde4_0_index_pattern",
Expand Down Expand Up @@ -344,6 +345,7 @@
"type": "metric",
"use_kibana_indexes": true
},
"title": "",
"type": "metrics",
"uiState": {}
}
Expand All @@ -358,7 +360,7 @@
"panelIndex": "dd7df777-8dae-4374-96ee-461be82fbde4",
"title": "StatefulSet Replicas Ready [Metrics Kubernetes]",
"type": "visualization",
"version": "8.4.0-SNAPSHOT"
"version": "8.6.0-SNAPSHOT"
},
{
"embeddableConfig": {
Expand Down Expand Up @@ -458,7 +460,7 @@
"panelIndex": "3e6790d6-de88-47de-8c3e-d8aa2c89c538",
"title": "StatefulSet Generation Observed [Metrics Kubernetes]",
"type": "visualization",
"version": "8.4.0-SNAPSHOT"
"version": "8.6.0-SNAPSHOT"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"version": "8.6.0-SNAPSHOT"
"version": "8.6.0"

},
{
"embeddableConfig": {
Expand Down Expand Up @@ -487,7 +489,7 @@
"drop_last_bucket": 0,
"filter": {
"language": "kuery",
"query": "event.dataset :\"kubernetes.state_statefulset\" "
"query": "data_stream.dataset :\"kubernetes.state_statefulset\" "
},
"id": "0db5e639-b021-4535-ab29-629ee94ad72a",
"index_pattern_ref_name": "metrics_34f2d226-f9e4-47f7-87d6-e3f5fb1db6d3_0_index_pattern",
Expand Down Expand Up @@ -544,6 +546,7 @@
"type": "metric",
"use_kibana_indexes": true
},
"title": "",
"type": "metrics",
"uiState": {}
}
Expand All @@ -558,7 +561,7 @@
"panelIndex": "34f2d226-f9e4-47f7-87d6-e3f5fb1db6d3",
"title": "StatefulSet Generation Desired [Metrics Kubernetes]",
"type": "visualization",
"version": "8.4.0-SNAPSHOT"
"version": "8.6.0-SNAPSHOT"
},
{
"embeddableConfig": {
Expand All @@ -571,13 +574,14 @@
},
{
"id": "metrics-*",
"name": "0ca4453d-a4eb-48d9-afb4-e70e141079f4",
"name": "33ac762e-904d-4a44-a943-e76d147b5770",
"type": "index-pattern"
}
],
"state": {
"adHocDataViews": {},
"datasourceStates": {
"indexpattern": {
"formBased": {
"layers": {
"0b03a29a-8bd6-485d-b34c-5682853a3ec6": {
"columnOrder": [
Expand Down Expand Up @@ -654,21 +658,22 @@
"meta": {
"alias": null,
"disabled": false,
"index": "0ca4453d-a4eb-48d9-afb4-e70e141079f4",
"key": "event.dataset",
"index": "33ac762e-904d-4a44-a943-e76d147b5770",
"key": "data_stream.dataset",
"negate": false,
"params": {
"query": "state_statefulset"
"query": "kubernetes.state_statefulset"
},
"type": "phrase"
},
"query": {
"match_phrase": {
"event.dataset": "kubernetes.state_statefulset"
"data_stream.dataset": "kubernetes.state_statefulset"
}
}
}
],
"internalReferences": [],
"query": {
"language": "kuery",
"query": ""
Expand Down Expand Up @@ -697,6 +702,8 @@
"valueLabels": "hide"
}
},
"title": "Replicas per StatefulSet [Metrics Kubernetes]",
"type": "lens",
"visualizationType": "lnsXY"
},
"enhancements": {},
Expand All @@ -712,17 +719,18 @@
"panelIndex": "f482071f-d956-4817-90b5-82a74d8aa841",
"title": "Replicas per StatefulSet [Metrics Kubernetes]",
"type": "lens",
"version": "8.4.0-SNAPSHOT"
"version": "8.6.0-SNAPSHOT"
}
],
"timeRestore": false,
"title": "[Metrics Kubernetes] StatefulSets",
"version": 1
},
"coreMigrationVersion": "8.4.0",
"coreMigrationVersion": "8.6.0",
"created_at": "2023-01-11T15:28:02.904Z",
"id": "kubernetes-21694370-bcb2-11ec-b64f-7dd6e8e82013",
"migrationVersion": {
"dashboard": "8.3.0"
"dashboard": "8.6.0"
},
"references": [
{
Expand Down Expand Up @@ -757,7 +765,7 @@
},
{
"id": "metrics-*",
"name": "f482071f-d956-4817-90b5-82a74d8aa841:0ca4453d-a4eb-48d9-afb4-e70e141079f4",
"name": "f482071f-d956-4817-90b5-82a74d8aa841:33ac762e-904d-4a44-a943-e76d147b5770",
"type": "index-pattern"
},
{
Expand Down
Loading