Podman socket won't show health status as a event #13493

floriplum · 2022-03-12T08:09:00Z

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description
The podman/docker socket won't update the health status of running containers.
That is important for services like traefik that check the container health.

Steps to reproduce the issue:

Start the podman socket
listen to the events on the socket.
curl --unix-socket /var/run/docker.sock http://localhost/events
Start a container that has a health check configured

Describe the results you received:
No update is visible when the container changes its state from starting to healthy

Describe the results you expected:
Getting an update when the container is healthy like you would get with docker.
{"status":"health_status: healthy","id":"59c5c31a8b6dfd30204b7fafafd2a413e65f53f4fd8a43bd17d5310667c19e00","from":"docker.io/gitlab/gitlab-ce:latest","Type":"container","Action":"health_status: healthy"

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Version:      3.4.0
API Version:  3.4.0
Go Version:   go1.16.8
Built:        Thu Sep 30 21:32:16 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers: []
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 4
  distribution:
    distribution: fedora
    variant: server
    version: "35"
  eventLogger: journald
  hostname: feddock1.fritz.box
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.14.10-300.fc35.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 518139904
  memTotal: 8325271552
  ociRuntime:
    name: crun
    package: crun-1.1-1.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.1
      commit: 5b341a145c4f515f96f55e3e7760d1c79ec3cf1f
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 8292397056
  swapTotal: 8324640768
  uptime: 14h 42m 59.74s (Approximately 0.58 days)
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/florian/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/florian/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 0
  runRoot: /run/user/1000/containers
  volumePath: /home/florian/.local/share/containers/storage/volumes
version:
  APIVersion: 3.4.0
  Built: 1633030336
  BuiltTime: Thu Sep 30 21:32:16 2021
  GitCommit: ""
  GoVersion: go1.16.8
  OsArch: linux/amd64
  Version: 3.4.0

Package info (e.g. output of rpm -q podman or apt list podman):

podman-3.4.0-1.fc35.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

The text was updated successfully, but these errors were encountered:

floriplum · 2022-03-12T08:11:05Z

I also opened an issue on the traefik github, but the issue looks like the podman api is different compared to docker for events.
traefik/traefik#8833

Marwel · 2022-03-12T10:22:23Z

When running "podman healthcheck run $containername", the health status is updated.
I also have other containers, where healthchecks are working as expected, so it is not always an issue.

floriplum · 2022-03-12T17:23:52Z

For me when i run podman healthcheck run $containername is still don't get the health status while listening to events in the socket.

mheon · 2022-03-16T17:54:43Z

Can you check if the events are showed in the podman events command, or if they are absent from both CLI and API?

floriplum · 2022-03-17T04:01:54Z

The events are basically the same for the socket and podman events.
It is missing for both the CLI and API.

Podman events output:

2022-03-17 04:56:43.276311453 +0100 CET image pull uptime-kuma
2022-03-17 04:56:43.433479516 +0100 CET volume create 02d8c44379839815426f0aa4e422e409fe7d45902965d3bdbd13f3f3c4783c93
2022-03-17 04:56:43.433577113 +0100 CET container history dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=, name=)
2022-03-17 04:56:43.452894279 +0100 CET container create dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson)
2022-03-17 04:56:43.64607344 +0100 CET container init dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson)
2022-03-17 04:56:43.671992434 +0100 CET container start dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson)
2022-03-17 04:56:43.672118967 +0100 CET container attach dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson)
2022-03-17 04:56:43.775732373 +0100 CET container exec dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson)
2022-03-17 04:56:43.979508506 +0100 CET container exec_died dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson, execID=1266561c93a0f230f85e2a0d82f8039caa3190d2a60c3fff0a64d49849a80bf7)
2022-03-17 04:57:45.032990833 +0100 CET container exec dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson)
2022-03-17 04:57:45.213075408 +0100 CET container exec_died dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson, execID=ff8a3ef938c52fb4b56ec09c5428e02acf8eb565ed5516f94c3c5a76766320a5)
2022-03-17 04:58:46.026469908 +0100 CET container exec dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson)
2022-03-17 04:58:46.209523927 +0100 CET container exec_died dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson, execID=37f69c31566496c5b42a5b2d95f9e5fa7b3b014b537cb05406603cdec556970b)
2022-03-17 04:59:47.021119165 +0100 CET container exec dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson)
2022-03-17 04:59:47.211663413 +0100 CET container exec_died dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136 (image=docker.io/louislam/uptime-kuma:latest, name=focused_robinson, execID=ea77f7cf5acd9848895f0def37489ca3951cc1049e0387f765eb60ce698119d4)

Socket output:

{"status":"pull","Type":"image","Action":"pull","Actor":{"ID":"","Attributes":{"containerExitCode":"0","image":"","name":"uptime-kuma"}},"scope":"local","time":1647489403,"timeNano":1647489403276311453}
{"status":"create","Type":"volume","Action":"create","Actor":{"ID":"","Attributes":{"containerExitCode":"0","image":"","name":"02d8c44379839815426f0aa4e422e409fe7d45902965d3bdbd13f3f3c4783c93"}},"scope":"local","time":1647489403,"timeNano":1647489403433479516}
{"status":"history","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Type":"container","Action":"history","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"0","image":"","name":""}},"scope":"local","time":1647489403,"timeNano":1647489403433577113}
{"status":"create","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","from":"docker.io/louislam/uptime-kuma:latest","Type":"container","Action":"create","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"0","image":"docker.io/louislam/uptime-kuma:latest","name":"focused_robinson"}},"scope":"local","time":1647489403,"timeNano":1647489403452894279}
{"status":"init","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","from":"docker.io/louislam/uptime-kuma:latest","Type":"container","Action":"init","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"0","image":"docker.io/louislam/uptime-kuma:latest","name":"focused_robinson"}},"scope":"local","time":1647489403,"timeNano":1647489403646073440}
{"status":"start","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","from":"docker.io/louislam/uptime-kuma:latest","Type":"container","Action":"start","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"0","image":"docker.io/louislam/uptime-kuma:latest","name":"focused_robinson"}},"scope":"local","time":1647489403,"timeNano":1647489403671992434}
{"status":"attach","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","from":"docker.io/louislam/uptime-kuma:latest","Type":"container","Action":"attach","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"0","image":"docker.io/louislam/uptime-kuma:latest","name":"focused_robinson"}},"scope":"local","time":1647489403,"timeNano":1647489403672118967}
{"status":"exec","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","from":"docker.io/louislam/uptime-kuma:latest","Type":"container","Action":"exec","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"0","image":"docker.io/louislam/uptime-kuma:latest","name":"focused_robinson"}},"scope":"local","time":1647489403,"timeNano":1647489403775732373}
{"status":"exec_died","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","from":"docker.io/louislam/uptime-kuma:latest","Type":"container","Action":"exec_died","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"1","execID":"1266561c93a0f230f85e2a0d82f8039caa3190d2a60c3fff0a64d49849a80bf7","image":"docker.io/louislam/uptime-kuma:latest","name":"focused_robinson"}},"scope":"local","time":1647489403,"timeNano":1647489403979508506}
{"status":"exec","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","from":"docker.io/louislam/uptime-kuma:latest","Type":"container","Action":"exec","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"0","image":"docker.io/louislam/uptime-kuma:latest","name":"focused_robinson"}},"scope":"local","time":1647489465,"timeNano":1647489465032990833}
{"status":"exec_died","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","from":"docker.io/louislam/uptime-kuma:latest","Type":"container","Action":"exec_died","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"0","execID":"ff8a3ef938c52fb4b56ec09c5428e02acf8eb565ed5516f94c3c5a76766320a5","image":"docker.io/louislam/uptime-kuma:latest","name":"focused_robinson"}},"scope":"local","time":1647489465,"timeNano":1647489465213075408}
{"status":"exec","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","from":"docker.io/louislam/uptime-kuma:latest","Type":"container","Action":"exec","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"0","image":"docker.io/louislam/uptime-kuma:latest","name":"focused_robinson"}},"scope":"local","time":1647489526,"timeNano":1647489526026469908}
{"status":"exec_died","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","from":"docker.io/louislam/uptime-kuma:latest","Type":"container","Action":"exec_died","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"0","execID":"37f69c31566496c5b42a5b2d95f9e5fa7b3b014b537cb05406603cdec556970b","image":"docker.io/louislam/uptime-kuma:latest","name":"focused_robinson"}},"scope":"local","time":1647489526,"timeNano":1647489526209523927}
{"status":"exec","id":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","from":"docker.io/louislam/uptime-kuma:latest","Type":"container","Action":"exec","Actor":{"ID":"dc668bf816be56f1f1990a995b2bff96e21596ecd9551a6d93bb81ddd3b98136","Attributes":{"containerExitCode":"0","image":"docker.io/louislam/uptime-kuma:latest","name":"focused_robinson"}},"scope":"local","time":1647489587,"timeNano":1647489587021119165}

mheon · 2022-03-17T14:31:00Z

Roger, thanks. Definitely looks like messages are not being generated.

We probably want to suppress the exec messages as well, as this is a healthcheck, not an exec (they use the same backend code, so I imagine that the exec events are firing instead?).

github-actions · 2022-04-17T00:06:38Z

A friendly reminder that this issue had no activity for 30 days.

github-actions · 2022-05-19T00:09:03Z

A friendly reminder that this issue had no activity for 30 days.

Omrigan · 2022-06-13T18:40:32Z

A friendly reminder that this issue had no activity for 30 days.

The issue is indeed present, and podman healthcheck run $containername indeed helps

mheon · 2022-06-13T19:30:19Z

@jakecorrenti This could be a good issue to look at

jakecorrenti · 2022-06-13T19:38:31Z

@mheon Do you want me to look at this one or #14493 first?

mheon · 2022-06-13T22:55:42Z

Your preference, though personally I'd recommend #14493 first

openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Mar 12, 2022

github-actions bot added the stale-issue label Apr 17, 2022

rhatdan removed the stale-issue label Apr 18, 2022

github-actions bot added the stale-issue label May 19, 2022

rhatdan removed the stale-issue label May 19, 2022

jakecorrenti mentioned this issue Jun 24, 2022

Show Health Status events #14705

Merged

openshift-ci bot closed this as completed in #14705 Jun 27, 2022

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 12, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Podman socket won't show health status as a event #13493

Podman socket won't show health status as a event #13493

floriplum commented Mar 12, 2022

floriplum commented Mar 12, 2022

Marwel commented Mar 12, 2022 •

edited

Loading

floriplum commented Mar 12, 2022

mheon commented Mar 16, 2022

floriplum commented Mar 17, 2022 •

edited

Loading

mheon commented Mar 17, 2022

github-actions bot commented Apr 17, 2022

github-actions bot commented May 19, 2022

Omrigan commented Jun 13, 2022

mheon commented Jun 13, 2022

jakecorrenti commented Jun 13, 2022

mheon commented Jun 13, 2022

Podman socket won't show health status as a event #13493

Podman socket won't show health status as a event #13493

Comments

floriplum commented Mar 12, 2022

floriplum commented Mar 12, 2022

Marwel commented Mar 12, 2022 • edited Loading

floriplum commented Mar 12, 2022

mheon commented Mar 16, 2022

floriplum commented Mar 17, 2022 • edited Loading

mheon commented Mar 17, 2022

github-actions bot commented Apr 17, 2022

github-actions bot commented May 19, 2022

Omrigan commented Jun 13, 2022

mheon commented Jun 13, 2022

jakecorrenti commented Jun 13, 2022

mheon commented Jun 13, 2022

Marwel commented Mar 12, 2022 •

edited

Loading

floriplum commented Mar 17, 2022 •

edited

Loading