Port IncludeAnnotations setting to Agent and small manifest fix

[ChrsMark]

What does this PR do?

Small followup fix related to #27691:

• Fix proposed manifest's condition from kubernetes.pod.labels.app == redis to ``kubernetes.labels.app == redis`
• Port IncludeAnnotations
• Handle start/stop of watchers properly

How to test this PR manually

Using the following redis Pod:

apiVersion: v1
kind: Pod
metadata:
  name: redis
  labels:
    role: main
    app: redis
  annotations:
    level: production
spec:
  hostNetwork: true
  dnsPolicy: ClusterFirstWithHostNet
  containers:
    - name: redis
      image: redis:latest
      ports:
        - name: web
          containerPort: 6379
          protocol: TCP

Test the following scenarios:

1. Check Pod's annotation is added as metadata

providers.kubernetes:
  scope: node
  kube_config: /Users/chrismark/.kube/config
  node: "kind-control-plane"
  cleanup_timeout: 360s
  include_annotations: ["level"]
  resources:
    pod:
      enabled: true

inputs:
- name: redis
  type: redis/metrics
  use_output: default
  meta:
    package:
      name: redis
      version: 0.3.6
  data_stream:
    namespace: default
  streams:
    - data_stream:
        dataset: redis.info
        type: metrics
      metricsets:
        - info
      hosts:
        - '${kubernetes.pod.ip}:${kubernetes.container.port}'
      idle_timeout: 20s
      maxconn: 10
      network: tcp
      period: 10s
      condition: ${kubernetes.labels.app} == 'redis' AND ${kubernetes.container.port_name} == 'web'

2. Check condition based on kubernetes.namespace_annotations.foo == bar

kubectl annotate ns default foo=bar

providers.kubernetes:
  scope: node
  kube_config: /Users/chrismark/.kube/config
  node: "kind-control-plane"
  cleanup_timeout: 360s
  include_annotations: ["level"]
  resources:
    pod:
      enabled: true

inputs:
- name: redis
  type: redis/metrics
  use_output: default
  meta:
    package:
      name: redis
      version: 0.3.6
  data_stream:
    namespace: default
  streams:
    - data_stream:
        dataset: redis.info
        type: metrics
      metricsets:
        - info
      hosts:
        - '${kubernetes.pod.ip}:${kubernetes.container.port}'
      idle_timeout: 20s
      maxconn: 10
      network: tcp
      period: 10s
      condition: ${kubernetes.namespace_annotations.foo} == 'bar' AND ${kubernetes.container.port_name} == 'web'

3. Check that ns annotations are being added to Pod's metadata.

providers.kubernetes:
 scope: node
 kube_config: /Users/chrismark/.kube/config
 node: "kind-control-plane"
 cleanup_timeout: 360s
 include_annotations: ["level"]
 add_resource_metadata:
    namespace:
      include_annotations: ["foo"]
 resources:
   pod:
     enabled: true

inputs:
- name: redis
 type: redis/metrics
 use_output: default
 meta:
   package:
     name: redis
     version: 0.3.6
 data_stream:
   namespace: default
 streams:
   - data_stream:
       dataset: redis.info
       type: metrics
     metricsets:
       - info
     hosts:
       - '${kubernetes.pod.ip}:${kubernetes.container.port}'
     idle_timeout: 20s
     maxconn: 10
     network: tcp
     period: 10s
     condition: ${kubernetes.labels.app} == 'redis' AND ${kubernetes.container.port_name} == 'web'

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Duration: 129 min 3 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

[MichaelKatsoulis]

LGTM with some minor cosmetic updates.