Events sometimes missing values for certain keys?

[zestysoft]

Describe the bug

Occasionally alerts will fire without some of the context that usually populates. When this happens these fields are empty:
container_image
container_image_tag
container_name
k8s_ns
k8s_pod_name

How to reproduce it

It's not clear why this happens.

Expected behaviour

Events always populate based on available data.

Here's an example of it not having complete information:

and here's an example of all the fields being populated:

• Falco version:
  0.37.1
• System info:
  two different node types:
  {
  "machine": "x86_64",
  "nodename": "falco-bdxmv",
  "release": "5.15.0-94-generic",
  "sysname": "Linux",
  "version": "# 104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024"
  }
  or
  {
  "machine": "aarch64",
  "nodename": "falco-hxtk9",
  "release": "5.15.0-1045-raspi",
  "sysname": "Linux",
  "version": "# 48-Ubuntu SMP PREEMPT Tue Jan 16 13:24:44 UTC 2024"
  }
• Cloud provider or hardware configuration:
  Locally hosted machines -- old mac laptops, raspberry pi's, repurposed pcs, vmware virtual machine
• OS:
  All nodes are running Ubuntu 220.4.3 LTS
• Kernel:
  All should be some flavor of 5.15
• Installation method:
  helm repo add falcosecurity https://falcosecurity.github.io/charts
  helm repo update
  helm install falco -n falco falcosecurity/falco -f values.yaml

values.yaml:

falcoctl:
  artifact:
    install:
      enabled: true
    follow:
      enabled: true
  config:
    artifact:
      install:
        refs: [falco-rules:3]
      follow:
        refs: [falco-rules:3]
falco:
  load_plugins: []
driver:
  kind: ebpf
tty: true
falcosidekick:
  enabled: true
  config:
    webhook:
      address: "<redacted test python script to troubleshoot this>"
    mattermost:
      webhookurl: "<redacted>"
customRules:
  just a bunch of rules to exempt detections

Additional context

I /think/ these have been limited to pods from daemonsets, but I'm not certain.

This screenshot is from a python script I whipped up to be a webhook destination so that I could rule out Mattermost:

and here's a raw json object of that event:

{'uuid': '20f5c131-abf5-4f77-be23-ecd0b3ae4e0a', 'output': '03:06:15.707626915: Notice Unexpected connection to K8s API Server from container (connection=172.17.201.3:48668->10.96.0.1:443 lport=443 rport=48668 fd_type=ipv4 fd_proto=fd.l4proto evt_type=connect user=root user_uid=0 user_loginuid=-1 process=csi-provisioner proc_exepath=/csi-provisioner parent=conmon command=csi-provisioner --csi-address=unix:///csi/csi-provisioner.sock --v=0 --timeout=2m30s --retry-interval-start=500ms --leader-election=true --leader-election-namespace=rook-ceph --leader-election-lease-duration=137s --leader-election-renew-deadline=107s --leader-election-retry-period=26s --default-fstype=ext4 --extra-create-metadata=true terminal=0 container_id=5769e14ce0a0 container_image=<NA> container_image_tag=<NA> container_name=<NA> k8s_ns=<NA> k8s_pod_name=<NA>)', 'priority': 'Notice', 'rule': 'Contact K8S API Server From Container', 'time': '2024-04-01T03:06:15.707626915Z', 'output_fields': {'container.id': '5769e14ce0a0', 'container.image.repository': None, 'container.image.tag': None, 'container.name': None, 'evt.time': 1711940775707626915, 'evt.type': 'connect', 'fd.lport': 443, 'fd.name': '172.17.201.3:48668->10.96.0.1:443', 'fd.rport': 48668, 'fd.type': 'ipv4', 'k8s.ns.name': None, 'k8s.pod.name': None, 'proc.cmdline': 'csi-provisioner --csi-address=unix:///csi/csi-provisioner.sock --v=0 --timeout=2m30s --retry-interval-start=500ms --leader-election=true --leader-election-namespace=rook-ceph --leader-election-lease-duration=137s --leader-election-renew-deadline=107s --leader-election-retry-period=26s --default-fstype=ext4 --extra-create-metadata=true', 'proc.exepath': '/csi-provisioner', 'proc.name': 'csi-provisioner', 'proc.pname': 'conmon', 'proc.tty': 0, 'user.loginuid': -1, 'user.name': 'root', 'user.uid': 0}, 'source': 'syscall', 'tags': ['T1565', 'container', 'k8s', 'maturity_stable', 'mitre_discovery', 'network'], 'hostname': 'falco-5pnrk'}

[zestysoft]

Ah, looks like y'all know about this already #2700