Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[8.17](backport #41903) Add regex pattern matching to add_kubernetes_metadata processor #42087

Merged
merged 1 commit into from
Dec 18, 2024
Merged

[8.17](backport #41903) Add regex pattern matching to add_kubernetes_metadata processor #42087

merged 1 commit into from
Dec 18, 2024
+115 −17

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Dec 17, 2024

Proposed commit message

Add the ability to extract the matching key from fields using a regex pattern to the add_kubernetes_metadata processor. I've added a new field - regex_pattern to the existing fields matcher. If the field isn't present, the matcher continues to work the same way it used to. The regex pattern has to contain a named capture group: key, and the first leftmost match for this group is the extracted key value. Other than this, the semantics are identical to the existing ones for the fields matcher.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

This was originally intended to fix https://github.com/elastic/ingest-dev/issues/3428, where we want to match on a part of system.process.cgroup.id. I've verified this by building an agent container image with this change enabled, and making the following changes to the default agent manifest:

system.hostfs: /hostfs
processors:
  - add_kubernetes_metadata:
      indexers:
        - container:
      matchers:
        - fields:
            lookup_fields: ['system.process.cgroup.id']
            regex_pattern: 'cri-containerd-(?P<key>[0-9a-z]+)\.scope'

With the Helm Chart, you can use the following values:

outputs:
  default:
    type: ESPlainAuthAPI
    url: ...
    api_key: ...

kubernetes:
  enabled: false

extraIntegrations:
  system:
    id: system-metrics
    preset: system
    type: system/metrics
    use_output: default
    hostfs: /hostfs
    meta:
      package:
        name: system
        version: 1.20.4
    data_stream:
      namespace: default
    streams:
      - data_stream:
          dataset: system.process
          type: metrics
        period: 10s
        processes:
          - .*
        process.include_top_n.by_cpu: 5
        process.include_top_n.by_memory: 5
        process.cmdline.cache.enabled: true
        process.cgroups.enabled: true
        process.include_cpu_ticks: false
        metricsets:
          - process
        system.hostfs: /hostfs
        processors:
          - add_kubernetes_metadata:
              indexers:
                - container:
              matchers:
                - fields:
                    lookup_fields: [ 'system.process.cgroup.id' ]
                    regex_pattern: 'cri-containerd-(?P<key>[0-9a-z]+)\.scope'

agent:
  presets:
    system:
      mode: daemonset
      securityContext:
        runAsUser: 0
        procMount: "Unmasked"
        privileged: true
        capabilities:
          add:
            - SYS_ADMIN
            - SYS_PTRACE
      serviceAccount:
        create: true
      clusterRole:
        create: true
      providers:
        kubernetes_leaderelection:
          enabled: false # disable leader election
      extraVolumes:
        - name: proc
          hostPath:
            path: /proc
        - name: cgroup
          hostPath:
            path: /sys/fs/cgroup
      extraVolumeMounts:
        - name: proc
          mountPath: /hostfs/proc
          readOnly: true
        - name: cgroup
          mountPath: /hostfs/sys/fs/cgroup
          readOnly: true

Related issues

Use cases

It's now possible to extract parts of fields for matching with kubernetes metadata. A use case which this is intended to make possible is adding kubernetes metadata to system process metrics. These metrics will contain the cgroup id under the field system.process.cgroup.id if cgroups are enabled, but the container id as reported by Kubernetes is only part of this value. In order to match it against the latter, we need the ability to only use a part of it.

Screenshots

Screenshot_20241205_113715

Example document 
{
  "_index": ".ds-metrics-system.process-default-2024.11.06-000003",
  "_id": "ntrqw3OUtFSFZegtAAABk5ZkekI",
  "_version": 1,
  "_score": 0,
  "_source": {
    "@timestamp": "2024-12-05T10:36:27.586Z",
    "agent": {
      "ephemeral_id": "45265d0f-a97b-49e9-a8cd-d80e55904374",
      "id": "38c65f92-5d57-4c2a-8b96-30dfa2eb7316",
      "name": "beats-control-plane",
      "type": "metricbeat",
      "version": "9.0.0"
    },
    "container": {
      "id": "8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2",
      "image": {
        "name": "docker.elastic.co/beats/elastic-agent:9.0.0-SNAPSHOT"
      },
      "runtime": "containerd"
    },
    "data_stream": {
      "dataset": "system.process",
      "namespace": "default",
      "type": "metrics"
    },
    "ecs": {
      "version": "8.0.0"
    },
    "elastic_agent": {
      "id": "38c65f92-5d57-4c2a-8b96-30dfa2eb7316",
      "snapshot": true,
      "version": "9.0.0"
    },
    "event": {
      "agent_id_status": "auth_metadata_missing",
      "dataset": "system.process",
      "duration": 108007040,
      "ingested": "2024-12-05T10:36:35Z",
      "module": "system"
    },
    "host": {
      "architecture": "x86_64",
      "containerized": true,
      "hostname": "beats-control-plane",
      "name": "beats-control-plane",
      "os": {
        "codename": "noble",
        "family": "debian",
        "kernel": "6.12.1-arch1-1",
        "name": "Ubuntu",
        "platform": "ubuntu",
        "type": "linux",
        "version": "24.04.1 LTS (Noble Numbat)"
      }
    },
    "kubernetes": {
      "container": {
        "name": "elastic-agent-standalone"
      },
      "daemonset": {
        "name": "elastic-agent-standalone"
      },
      "labels": {
        "app_kubernetes_io/name": "elastic-agent-standalone",
        "controller-revision-hash": "898fff5b7",
        "pod-template-generation": "7"
      },
      "namespace": "kube-system",
      "namespace_labels": {
        "kubernetes_io/metadata_name": "kube-system"
      },
      "namespace_uid": "0a414bbf-1241-4d5e-bb5d-00c52330c6f6",
      "node": {
        "hostname": "beats-control-plane",
        "labels": {
          "beta_kubernetes_io/arch": "amd64",
          "beta_kubernetes_io/os": "linux",
          "kubernetes_io/arch": "amd64",
          "kubernetes_io/hostname": "beats-control-plane",
          "kubernetes_io/os": "linux",
          "node-role_kubernetes_io/control-plane": ""
        },
        "name": "beats-control-plane",
        "uid": "ce0fd5af-a553-4b93-9a3a-78e76bc0c861"
      },
      "pod": {
        "ip": "172.18.0.2",
        "name": "elastic-agent-standalone-c5cl4",
        "uid": "fa935f6e-f760-4277-85a6-aa1b40d3441d"
      }
    },
    "metricset": {
      "name": "process",
      "period": 10000
    },
    "orchestrator": {
      "cluster": {
        "name": "beats",
        "url": "beats-control-plane:6443"
      }
    },
    "process": {
      "args": [
        "-c",
        "-e",
        "/etc/elastic-agent/agent.yml",
        "container",
        "elastic-agent"
      ],
      "command_line": "elastic-agent container -c /etc/elastic-agent/agent.yml -e",
      "cpu": {
        "pct": 0,
        "start_time": "2024-12-05T10:28:43.000Z"
      },
      "executable": "/usr/share/elastic-agent/data/elastic-agent-6eb929/elastic-agent",
      "memory": {
        "pct": 0.007
      },
      "name": "elastic-agent",
      "parent": {
        "pid": 22388
      },
      "pgid": 22407,
      "pid": 22407,
      "state": "sleeping",
      "working_directory": "/usr/share/elastic-agent"
    },
    "service": {
      "type": "system"
    },
    "system": {
      "process": {
        "cgroup": {
          "cgroups_version": 2,
          "cpu": {
            "id": "cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope",
            "path": "/kubelet.slice/kubelet-kubepods.slice/kubelet-kubepods-burstable.slice/kubelet-kubepods-burstable-podfa935f6e_f760_4277_85a6_aa1b40d3441d.slice/cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope",
            "pressure": {
              "full": {
                "10": {
                  "pct": 0
                },
                "60": {
                  "pct": 0
                },
                "300": {
                  "pct": 0
                },
                "total": 18170
              },
              "some": {
                "10": {
                  "pct": 0
                },
                "60": {
                  "pct": 0
                },
                "300": {
                  "pct": 0
                },
                "total": 21133
              }
            },
            "stats": {
              "periods": 0,
              "system": {
                "norm": {
                  "pct": 0
                },
                "ns": 3280592,
                "pct": 0
              },
              "throttled": {
                "periods": 0,
                "us": 0
              },
              "usage": {
                "norm": {
                  "pct": 0
                },
                "ns": 14528746,
                "pct": 0
              },
              "user": {
                "norm": {
                  "pct": 0
                },
                "ns": 11248154,
                "pct": 0
              }
            }
          },
          "id": "cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope",
          "io": {
            "id": "cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope",
            "path": "/kubelet.slice/kubelet-kubepods.slice/kubelet-kubepods-burstable.slice/kubelet-kubepods-burstable-podfa935f6e_f760_4277_85a6_aa1b40d3441d.slice/cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope",
            "pressure": {
              "full": {
                "10": {
                  "pct": 0
                },
                "60": {
                  "pct": 0
                },
                "300": {
                  "pct": 0
                },
                "total": 2501
              },
              "some": {
                "10": {
                  "pct": 0
                },
                "60": {
                  "pct": 0
                },
                "300": {
                  "pct": 0
                },
                "total": 2537
              }
            },
            "stats": {
              "dm-0": {
                "discarded": {
                  "bytes": 0,
                  "ios": 0
                },
                "read": {
                  "bytes": 29102080,
                  "ios": 370
                },
                "write": {
                  "bytes": 692224,
                  "ios": 78
                }
              },
              "nvme0n1": {
                "discarded": {
                  "bytes": 0,
                  "ios": 0
                },
                "read": {
                  "bytes": 29102080,
                  "ios": 370
                },
                "write": {
                  "bytes": 0,
                  "ios": 0
                }
              }
            }
          },
          "memory": {
            "id": "cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope",
            "mem": {
              "events": {
                "high": 0,
                "low": 0,
                "max": 0,
                "oom": 0,
                "oom_kill": 0
              },
              "low": {
                "bytes": 0
              },
              "max": {
                "bytes": 1073741824
              },
              "usage": {
                "bytes": 833597440
              }
            },
            "memsw": {
              "events": {
                "fail": 0,
                "high": 0,
                "max": 0
              },
              "low": {
                "bytes": 0
              },
              "max": {
                "bytes": 0
              },
              "usage": {
                "bytes": 0
              }
            },
            "path": "/kubelet.slice/kubelet-kubepods.slice/kubelet-kubepods-burstable.slice/kubelet-kubepods-burstable-podfa935f6e_f760_4277_85a6_aa1b40d3441d.slice/cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope",
            "stats": {
              "active_anon": {
                "bytes": 786595840
              },
              "active_file": {
                "bytes": 25976832
              },
              "anon": {
                "bytes": 700223488
              },
              "anon_thp": {
                "bytes": 218103808
              },
              "file": {
                "bytes": 29110272
              },
              "file_dirty": {
                "bytes": 0
              },
              "file_mapped": {
                "bytes": 282624
              },
              "file_thp": {
                "bytes": 0
              },
              "file_writeback": {
                "bytes": 0
              },
              "htp_collapse_alloc": 0,
              "inactive_anon": {
                "bytes": 36864
              },
              "inactive_file": {
                "bytes": 3133440
              },
              "kernel_stack": {
                "bytes": 3227648
              },
              "major_page_faults": 44,
              "page_activate": 0,
              "page_deactivate": 0,
              "page_faults": 154795,
              "page_lazy_free": 0,
              "page_lazy_freed": 0,
              "page_refill": 7,
              "page_scan": 128,
              "page_steal": 128,
              "page_tables": {
                "bytes": 8769536
              },
              "per_cpu": {
                "bytes": 8712
              },
              "shmem": {
                "bytes": 0
              },
              "shmem_thp": {
                "bytes": 0
              },
              "slab": {
                "bytes": 4922512
              },
              "slab_reclaimable": {
                "bytes": 2445544
              },
              "slab_unreclaimable": {
                "bytes": 2476968
              },
              "sock": {
                "bytes": 0
              },
              "swap_cached": {
                "bytes": 0
              },
              "thp_fault_alloc": 183,
              "unevictable": {
                "bytes": 0
              },
              "workingset_activate_anon": 0,
              "workingset_activate_file": 0,
              "workingset_node_reclaim": 0,
              "workingset_refault_anon": 0,
              "workingset_refault_file": 0,
              "workingset_restore_anon": 0,
              "workingset_restore_file": 0
            }
          },
          "path": "/kubelet.slice/kubelet-kubepods.slice/kubelet-kubepods-burstable.slice/kubelet-kubepods-burstable-podfa935f6e_f760_4277_85a6_aa1b40d3441d.slice/cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope"
        },
        "cmdline": "elastic-agent container -c /etc/elastic-agent/agent.yml -e",
        "cpu": {
          "start_time": "2024-12-05T10:28:43.000Z",
          "total": {
            "norm": {
              "pct": 0
            },
            "pct": 0,
            "value": 1680
          }
        },
        "fd": {
          "limit": {
            "hard": 1073741816,
            "soft": 1073741816
          },
          "open": 35
        },
        "io": {
          "cancelled_write_bytes": 0,
          "read_bytes": 208896,
          "read_char": 6200556,
          "read_ops": 4769,
          "write_bytes": 552960,
          "write_char": 1320522,
          "write_ops": 1839
        },
        "memory": {
          "rss": {
            "bytes": 241573888,
            "pct": 0.007
          },
          "share": 151502848,
          "size": 4001886208
        },
        "num_threads": 31,
        "state": "sleeping"
      }
    },
    "user": {
      "name": "root"
    }
  },
  "fields": {
    "kubernetes.node.uid": [
      "ce0fd5af-a553-4b93-9a3a-78e76bc0c861"
    ],
    "orchestrator.cluster.name": [
      "beats"
    ],
    "system.process.cgroup.memory.mem.events.oom_kill": [
      0
    ],
    "elastic_agent.version": [
      "9.0.0"
    ],
    "kubernetes.namespace_uid": [
      "0a414bbf-1241-4d5e-bb5d-00c52330c6f6"
    ],
    "system.process.cgroup.memory.stats.page_deactivate": [
      0
    ],
    "host.os.name.text": [
      "Ubuntu"
    ],
    "system.process.cgroup.cpu.pressure.full.total": [
      18170
    ],
    "host.hostname": [
      "beats-control-plane"
    ],
    "kubernetes.node.labels.kubernetes_io/os": [
      "linux"
    ],
    "container.id": [
      "8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2"
    ],
    "system.process.cgroup.io.pressure.some.60.pct": [
      0
    ],
    "service.type": [
      "system"
    ],
    "system.process.cgroup.io.pressure.some.total": [
      2537
    ],
    "host.os.version": [
      "24.04.1 LTS (Noble Numbat)"
    ],
    "system.process.cgroup.memory.stats.file_dirty.bytes": [
      0
    ],
    "kubernetes.node.labels.beta_kubernetes_io/os": [
      "linux"
    ],
    "system.process.cgroup.memory.stats.page_activate": [
      0
    ],
    "system.process.cgroup.io.stats.nvme0n1.read.bytes": [
      29102080
    ],
    "system.process.cgroup.memory.mem.events.low": [
      0
    ],
    "system.process.cgroup.cpu.stats.throttled.periods": [
      0
    ],
    "metricset.name.text": [
      "process"
    ],
    "kubernetes.labels.app_kubernetes_io/name": [
      "elastic-agent-standalone"
    ],
    "system.process.cgroup.memory.mem.low.bytes": [
      0
    ],
    "system.process.cgroup.cpu.stats.system.ns": [
      3280592
    ],
    "host.os.type": [
      "linux"
    ],
    "system.process.cgroup.memory.stats.htp_collapse_alloc": [
      0
    ],
    "system.process.cgroup.memory.mem.events.max": [
      0
    ],
    "kubernetes.node.name.text": [
      "beats-control-plane"
    ],
    "host.containerized": [
      true
    ],
    "system.process.cmdline": [
      "elastic-agent container -c /etc/elastic-agent/agent.yml -e"
    ],
    "process.command_line.text": [
      "elastic-agent container -c /etc/elastic-agent/agent.yml -e"
    ],
    "process.memory.pct": [
      0.007
    ],
    "kubernetes.node.hostname": [
      "beats-control-plane"
    ],
    "system.process.cgroup.cpu.stats.usage.pct": [
      0
    ],
    "system.process.cgroup.memory.stats.workingset_refault_file": [
      0
    ],
    "process.working_directory": [
      "/usr/share/elastic-agent"
    ],
    "system.process.io.read_char": [
      6200556
    ],
    "agent.type": [
      "metricbeat"
    ],
    "process.executable.text": [
      "/usr/share/elastic-agent/data/elastic-agent-6eb929/elastic-agent"
    ],
    "system.process.io.write_bytes": [
      552960
    ],
    "system.process.cgroup.memory.stats.swap_cached.bytes": [
      0
    ],
    "kubernetes.pod.ip": [
      "172.18.0.2"
    ],
    "system.process.num_threads": [
      31
    ],
    "system.process.cgroup.memory.stats.workingset_node_reclaim": [
      0
    ],
    "kubernetes.container.name": [
      "elastic-agent-standalone"
    ],
    "elastic_agent.id": [
      "38c65f92-5d57-4c2a-8b96-30dfa2eb7316"
    ],
    "process.working_directory.text": [
      "/usr/share/elastic-agent"
    ],
    "system.process.io.write_char": [
      1320522
    ],
    "metricset.period": [
      10000
    ],
    "system.process.cgroup.memory.memsw.events.max": [
      0
    ],
    "system.process.cgroup.memory.stats.workingset_activate_file": [
      0
    ],
    "system.process.cgroup.io.pressure.full.10.pct": [
      0
    ],
    "system.process.cgroup.io.stats.nvme0n1.read.ios": [
      370
    ],
    "kubernetes.node.labels.beta_kubernetes_io/arch": [
      "amd64"
    ],
    "host.os.platform": [
      "ubuntu"
    ],
    "system.process.cgroup.cpu.pressure.full.60.pct": [
      0
    ],
    "system.process.io.read_ops": [
      4769
    ],
    "kubernetes.node.labels.kubernetes_io/arch": [
      "amd64"
    ],
    "container.image.name.text": [
      "docker.elastic.co/beats/elastic-agent:9.0.0-SNAPSHOT"
    ],
    "user.name.text": [
      "root"
    ],
    "system.process.cgroup.memory.stats.workingset_restore_file": [
      0
    ],
    "system.process.cgroup.memory.memsw.low.bytes": [
      0
    ],
    "process.name.text": [
      "elastic-agent"
    ],
    "system.process.cgroup.io.stats.dm-0.discarded.bytes": [
      0
    ],
    "system.process.cgroup.memory.stats.anon_thp.bytes": [
      218103808
    ],
    "system.process.cgroup.cpu.path": [
      "/kubelet.slice/kubelet-kubepods.slice/kubelet-kubepods-burstable.slice/kubelet-kubepods-burstable-podfa935f6e_f760_4277_85a6_aa1b40d3441d.slice/cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope"
    ],
    "system.process.cgroup.memory.stats.page_lazy_freed": [
      0
    ],
    "system.process.cgroup.memory.stats.file_thp.bytes": [
      0
    ],
    "system.process.memory.size": [
      4001886208
    ],
    "host.os.name": [
      "Ubuntu"
    ],
    "system.process.cgroup.cpu.pressure.some.300.pct": [
      0
    ],
    "system.process.cgroup.io.stats.nvme0n1.write.ios": [
      0
    ],
    "kubernetes.container.name.text": [
      "elastic-agent-standalone"
    ],
    "process.name": [
      "elastic-agent"
    ],
    "system.process.cgroup.memory.stats.workingset_refault_anon": [
      0
    ],
    "agent.version": [
      "9.0.0"
    ],
    "host.os.family": [
      "debian"
    ],
    "system.process.cgroup.cpu.stats.user.norm.pct": [
      0
    ],
    "system.process.cgroup.memory.stats.inactive_anon.bytes": [
      36864
    ],
    "kubernetes.pod.name.text": [
      "elastic-agent-standalone-c5cl4"
    ],
    "system.process.cgroup.memory.stats.major_page_faults": [
      44
    ],
    "system.process.fd.limit.hard": [
      1073741816
    ],
    "system.process.cgroup.memory.stats.slab.bytes": [
      4922512
    ],
    "system.process.cgroup.path": [
      "/kubelet.slice/kubelet-kubepods.slice/kubelet-kubepods-burstable.slice/kubelet-kubepods-burstable-podfa935f6e_f760_4277_85a6_aa1b40d3441d.slice/cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope"
    ],
    "system.process.cgroup.memory.stats.file_writeback.bytes": [
      0
    ],
    "event.module": [
      "system"
    ],
    "system.process.io.write_ops": [
      1839
    ],
    "system.process.cgroup.io.stats.dm-0.discarded.ios": [
      0
    ],
    "host.os.kernel": [
      "6.12.1-arch1-1"
    ],
    "system.process.cgroup.io.stats.dm-0.read.bytes": [
      29102080
    ],
    "kubernetes.pod.name": [
      "elastic-agent-standalone-c5cl4"
    ],
    "system.process.cgroup.cpu.pressure.some.10.pct": [
      0
    ],
    "system.process.cpu.start_time": [
      "2024-12-05T10:28:43.000Z"
    ],
    "process.args": [
      "-c",
      "-e",
      "/etc/elastic-agent/agent.yml",
      "container",
      "elastic-agent"
    ],
    "system.process.cgroup.memory.stats.page_scan": [
      128
    ],
    "kubernetes.namespace_labels.kubernetes_io/metadata_name": [
      "kube-system"
    ],
    "system.process.cgroup.id": [
      "cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope"
    ],
    "system.process.cgroup.io.pressure.full.total": [
      2501
    ],
    "system.process.cgroup.io.stats.dm-0.write.bytes": [
      692224
    ],
    "system.process.cpu.total.pct": [
      0
    ],
    "kubernetes.node.labels.kubernetes_io/hostname": [
      "beats-control-plane"
    ],
    "system.process.cgroup.cpu.stats.usage.ns": [
      14528746
    ],
    "system.process.state": [
      "sleeping"
    ],
    "system.process.cgroup.io.path": [
      "/kubelet.slice/kubelet-kubepods.slice/kubelet-kubepods-burstable.slice/kubelet-kubepods-burstable-podfa935f6e_f760_4277_85a6_aa1b40d3441d.slice/cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope"
    ],
    "system.process.cgroup.memory.mem.events.high": [
      0
    ],
    "system.process.cgroup.memory.stats.per_cpu.bytes": [
      8712
    ],
    "system.process.memory.rss.pct": [
      0.007
    ],
    "system.process.cpu.total.value": [
      1680
    ],
    "process.parent.pid": [
      22388
    ],
    "system.process.cgroup.io.pressure.full.60.pct": [
      0
    ],
    "system.process.cgroup.io.stats.nvme0n1.discarded.ios": [
      0
    ],
    "container.image.name": [
      "docker.elastic.co/beats/elastic-agent:9.0.0-SNAPSHOT"
    ],
    "agent.name.text": [
      "beats-control-plane"
    ],
    "system.process.cgroup.cpu.stats.periods": [
      0
    ],
    "system.process.fd.open": [
      35
    ],
    "agent.name": [
      "beats-control-plane"
    ],
    "event.agent_id_status": [
      "auth_metadata_missing"
    ],
    "system.process.cgroup.memory.stats.slab_unreclaimable.bytes": [
      2476968
    ],
    "system.process.cgroup.memory.memsw.events.high": [
      0
    ],
    "system.process.cgroup.io.pressure.full.300.pct": [
      0
    ],
    "system.process.cgroup.memory.stats.anon.bytes": [
      700223488
    ],
    "system.process.cgroup.memory.stats.page_lazy_free": [
      0
    ],
    "host.architecture": [
      "x86_64"
    ],
    "system.process.cgroup.cpu.pressure.some.total": [
      21133
    ],
    "system.process.cgroup.cpu.stats.user.ns": [
      11248154
    ],
    "system.process.cgroup.memory.stats.workingset_restore_anon": [
      0
    ],
    "agent.id": [
      "38c65f92-5d57-4c2a-8b96-30dfa2eb7316"
    ],
    "system.process.cgroup.memory.stats.workingset_activate_anon": [
      0
    ],
    "system.process.cgroup.memory.stats.active_anon.bytes": [
      786595840
    ],
    "kubernetes.node.labels.node-role_kubernetes_io/control-plane": [
      ""
    ],
    "system.process.cgroup.memory.stats.active_file.bytes": [
      25976832
    ],
    "user.name": [
      "root"
    ],
    "system.process.cgroup.io.id": [
      "cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope"
    ],
    "system.process.cgroup.memory.mem.usage.bytes": [
      833597440
    ],
    "system.process.cgroup.memory.memsw.usage.bytes": [
      0
    ],
    "elastic_agent.snapshot": [
      true
    ],
    "system.process.cgroup.cpu.pressure.full.300.pct": [
      0
    ],
    "system.process.cgroup.cpu.stats.user.pct": [
      0
    ],
    "system.process.cgroup.io.pressure.some.10.pct": [
      0
    ],
    "host.os.codename": [
      "noble"
    ],
    "system.process.cgroup.cpu.stats.system.norm.pct": [
      0
    ],
    "system.process.io.cancelled_write_bytes": [
      0
    ],
    "kubernetes.labels.pod-template-generation": [
      "7"
    ],
    "system.process.memory.share": [
      151502848
    ],
    "process.pgid": [
      22407
    ],
    "event.duration": [
      108007040
    ],
    "system.process.cgroup.cpu.pressure.some.60.pct": [
      0
    ],
    "system.process.memory.rss.bytes": [
      241573888
    ],
    "event.ingested": [
      "2024-12-05T10:36:35.000Z"
    ],
    "orchestrator.cluster.name.text": [
      "beats"
    ],
    "@timestamp": [
      "2024-12-05T10:36:27.586Z"
    ],
    "data_stream.dataset": [
      "system.process"
    ],
    "system.process.cgroup.memory.stats.file_mapped.bytes": [
      282624
    ],
    "agent.ephemeral_id": [
      "45265d0f-a97b-49e9-a8cd-d80e55904374"
    ],
    "system.process.cgroup.memory.stats.unevictable.bytes": [
      0
    ],
    "process.cpu.pct": [
      0
    ],
    "system.process.cgroup.memory.stats.shmem.bytes": [
      0
    ],
    "system.process.cpu.total.norm.pct": [
      0
    ],
    "kubernetes.daemonset.name": [
      "elastic-agent-standalone"
    ],
    "system.process.cgroup.memory.stats.page_tables.bytes": [
      8769536
    ],
    "process.pid": [
      22407
    ],
    "system.process.cgroup.io.pressure.some.300.pct": [
      0
    ],
    "system.process.cgroup.memory.stats.page_steal": [
      128
    ],
    "kubernetes.namespace": [
      "kube-system"
    ],
    "system.process.fd.limit.soft": [
      1073741816
    ],
    "host.name": [
      "beats-control-plane"
    ],
    "system.process.cgroup.cpu.id": [
      "cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope"
    ],
    "system.process.cgroup.io.stats.dm-0.read.ios": [
      370
    ],
    "system.process.cgroup.memory.stats.kernel_stack.bytes": [
      3227648
    ],
    "kubernetes.daemonset.name.text": [
      "elastic-agent-standalone"
    ],
    "data_stream.type": [
      "metrics"
    ],
    "system.process.cgroup.memory.memsw.events.fail": [
      0
    ],
    "system.process.cgroup.memory.stats.page_refill": [
      7
    ],
    "container.runtime": [
      "containerd"
    ],
    "system.process.cgroup.memory.memsw.max.bytes": [
      0
    ],
    "ecs.version": [
      "8.0.0"
    ],
    "system.process.cgroup.memory.mem.max.bytes": [
      1073741824
    ],
    "system.process.cgroup.io.stats.nvme0n1.write.bytes": [
      0
    ],
    "system.process.cgroup.memory.stats.shmem_thp.bytes": [
      0
    ],
    "system.process.cgroup.cpu.stats.system.pct": [
      0
    ],
    "kubernetes.node.name": [
      "beats-control-plane"
    ],
    "system.process.cgroup.memory.id": [
      "cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope"
    ],
    "system.process.cgroup.memory.mem.events.oom": [
      0
    ],
    "system.process.cgroup.cpu.stats.throttled.us": [
      0
    ],
    "kubernetes.pod.uid": [
      "fa935f6e-f760-4277-85a6-aa1b40d3441d"
    ],
    "system.process.io.read_bytes": [
      208896
    ],
    "process.state": [
      "sleeping"
    ],
    "system.process.cgroup.memory.stats.file.bytes": [
      29110272
    ],
    "orchestrator.cluster.url": [
      "beats-control-plane:6443"
    ],
    "system.process.cgroup.memory.path": [
      "/kubelet.slice/kubelet-kubepods.slice/kubelet-kubepods-burstable.slice/kubelet-kubepods-burstable-podfa935f6e_f760_4277_85a6_aa1b40d3441d.slice/cri-containerd-8aaaebb734b3d2da523d20805499164e227f914bcc1c9cbed2c37a1a803cb1e2.scope"
    ],
    "process.executable": [
      "/usr/share/elastic-agent/data/elastic-agent-6eb929/elastic-agent"
    ],
    "system.process.cgroup.cpu.pressure.full.10.pct": [
      0
    ],
    "data_stream.namespace": [
      "default"
    ],
    "system.process.cgroup.cpu.stats.usage.norm.pct": [
      0
    ],
    "system.process.cgroup.memory.stats.sock.bytes": [
      0
    ],
    "system.process.cgroup.memory.stats.slab_reclaimable.bytes": [
      2445544
    ],
    "system.process.cgroup.memory.stats.thp_fault_alloc": [
      183
    ],
    "system.process.cgroup.cgroups_version": [
      2
    ],
    "metricset.name": [
      "process"
    ],
    "process.cpu.start_time": [
      "2024-12-05T10:28:43.000Z"
    ],
    "system.process.cgroup.io.stats.nvme0n1.discarded.bytes": [
      0
    ],
    "system.process.cgroup.memory.stats.inactive_file.bytes": [
      3133440
    ],
    "kubernetes.labels.controller-revision-hash": [
      "898fff5b7"
    ],
    "process.command_line": [
      "elastic-agent container -c /etc/elastic-agent/agent.yml -e"
    ],
    "system.process.cgroup.io.stats.dm-0.write.ios": [
      78
    ],
    "event.dataset": [
      "system.process"
    ],
    "system.process.cgroup.memory.stats.page_faults": [
      154795
    ]
  }
}
This is an automatic backport of pull request #41903 done by [Mergify](https://mergify.com).

@swiatekm @mergify
Add regex pattern matching to add_kubernetes_metadata processor (#41903)
a821686 
* Add regex pattern matching to add_kubernetes_metadata processor

* Add changelog entry

* Add documentation

* Fix changelog

(cherry picked from commit da7bbf6)
@mergify mergify bot requested a review from a team as a code owner December 17, 2024 15:47
@mergify mergify bot added the backport label Dec 17, 2024
@mergify mergify bot requested review from rdner and mauri870 and removed request for a team December 17, 2024 15:47
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Dec 17, 2024
@botelastic
Copy link

botelastic bot commented Dec 17, 2024

This pull request doesn't have a Team:<team> label.

@swiatekm swiatekm merged commit 477e041 into 8.17 Dec 18, 2024
142 checks passed
@swiatekm swiatekm deleted the mergify/bp/8.17/pr-41903 branch December 18, 2024 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@swiatekm swiatekm swiatekm approved these changes

@rdner rdner Awaiting requested review from rdner rdner is a code owner automatically assigned from elastic/elastic-agent-data-plane

@mauri870 mauri870 Awaiting requested review from mauri870 mauri870 is a code owner automatically assigned from elastic/elastic-agent-data-plane

Assignees

@swiatekm swiatekm

Labels
backport needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@swiatekm