Podman not working with cron #7242
Labels
kind/bug
Categorizes issue or PR as related to a bug.
locked - please file new issue/PR
Assist humans wanting to comment on an old issue or PR with locked comments.
Comments
openshift-ci-robot
added
the
kind/bug
|
You need to run 'loginctl enable-linger' on the user. We rely on several
Systemd managed temporary directories which are only created when the user
logs in, but unfortunately Cron running something does not count as a login
so they are not there and Podman will fail. Enabling linger will have
Systemd keep the directories around and should resolve the issue.
…On Thu, Aug 6, 2020, 03:58 dzerhusen ***@***.***> wrote:
/kind bug
*Description*
Podman not working with cron
*Steps to reproduce the issue:*
1.Edit as cadmin User crontab -e
50 * * * * /opt/scripts/backup/postgresql_backup.sh >>
/opt/logs/backup/postgresql_backup.log 2>&1
1. vi /opt/scripts/backup/postgresql_backup.sh:
#!/bin/bash
/bin/podman exec --log-level=debug postgresql /usr/bin/psql -U postgres -l
1. wait and look into the log
*Describe the results you received:*
podman command is working fine on a normal terminal. I got the result of
the psql-command. See below.
I also tried setting the same env vars from my local terminal into the
bash script. Problem still exists.
I tried setting the path env var in the script.
*Describe the results you expected:*
Podman cronjob is succeeding
Versions
***@***.***~]$ cat /etc/redhat-release
CentOS Linux release 8.2.2004 (Core)
***@***.*** ~]$ podman version
Version: 2.0.4
API Version: 1
Go Version: go1.13.4
Built: Thu Jan 1 01:00:00 1970
OS/Arch: linux/amd64
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c08dc8d38d4c docker-registry.lmis.de/wm-services/postgresql:12 postgres 24 hours ago Up 24 hours ago 0.0.0.0:5432->5432/tcp postgresql
**Cronjob Results:**
***@***.*** ~]$ cat /opt/logs/backup/postgresql_backup.log
cadmin
time="2020-08-06T09:50:01+02:00" level=info msg="/bin/podman filtering at log level debug"
time="2020-08-06T09:50:01+02:00" level=debug msg="Called exec.PersistentPreRunE(/bin/podman exec --log-level=debug postgresql /usr/bin/psql -U postgres -l)"
time="2020-08-06T09:50:01+02:00" level=debug msg="Ignoring libpod.conf EventsLogger setting \"/home/cadmin/.config/containers/containers.conf\". Use \"journald\" if you want to change this setting and remove libpod.conf files."
time="2020-08-06T09:50:01+02:00" level=debug msg="Using conmon: \"/usr/bin/conmon\""
time="2020-08-06T09:50:01+02:00" level=debug msg="Initializing boltdb state at /home/cadmin/.local/share/containers/storage/libpod/bolt_state.db"
time="2020-08-06T09:50:01+02:00" level=debug msg="Overriding tmp dir \"/run/user/1002/libpod/tmp\" with \"/tmp/run-1002/libpod/tmp\" from database"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using graph driver overlay"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using graph root /home/cadmin/.local/share/containers/storage"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using run root /tmp/run-1002"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using static dir /home/cadmin/.local/share/containers/storage/libpod"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using tmp dir /tmp/run-1002/libpod/tmp"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using volume path /home/cadmin/.local/share/containers/storage/volumes"
time="2020-08-06T09:50:01+02:00" level=debug msg="Set libpod namespace to \"\""
time="2020-08-06T09:50:01+02:00" level=debug msg="Not configuring container store"
time="2020-08-06T09:50:01+02:00" level=debug msg="Initializing event backend file"
time="2020-08-06T09:50:01+02:00" level=debug msg="using runtime \"/usr/bin/runc\""
time="2020-08-06T09:50:01+02:00" level=warning msg="Error initializing configured OCI runtime crun: no valid executable found for OCI runtime crun: invalid argument"
time="2020-08-06T09:50:01+02:00" level=warning msg="Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument"
time="2020-08-06T09:50:01+02:00" level=info msg="Setting parallel job count to 13"
time="2020-08-06T09:50:01+02:00" level=info msg="/bin/podman filtering at log level debug"
time="2020-08-06T09:50:01+02:00" level=debug msg="Called exec.PersistentPreRunE(/bin/podman exec --log-level=debug postgresql /usr/bin/psql -U postgres -l)"
time="2020-08-06T09:50:01+02:00" level=debug msg="Ignoring libpod.conf EventsLogger setting \"/home/cadmin/.config/containers/containers.conf\". Use \"journald\" if you want to change this setting and remove libpod.conf files."
time="2020-08-06T09:50:01+02:00" level=debug msg="Using conmon: \"/usr/bin/conmon\""
time="2020-08-06T09:50:01+02:00" level=debug msg="Initializing boltdb state at /home/cadmin/.local/share/containers/storage/libpod/bolt_state.db"
time="2020-08-06T09:50:01+02:00" level=debug msg="Overriding tmp dir \"/run/user/1002/libpod/tmp\" with \"/tmp/run-1002/libpod/tmp\" from database"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using graph driver overlay"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using graph root /home/cadmin/.local/share/containers/storage"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using run root /tmp/run-1002"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using static dir /home/cadmin/.local/share/containers/storage/libpod"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using tmp dir /tmp/run-1002/libpod/tmp"
time="2020-08-06T09:50:01+02:00" level=debug msg="Using volume path /home/cadmin/.local/share/containers/storage/volumes"
time="2020-08-06T09:50:01+02:00" level=debug msg="Set libpod namespace to \"\""
time="2020-08-06T09:50:01+02:00" level=debug msg="[graphdriver] trying provided driver \"overlay\""
time="2020-08-06T09:50:01+02:00" level=debug msg="overlay: mount_program=/bin/fuse-overlayfs"
time="2020-08-06T09:50:01+02:00" level=debug msg="overlay: mount_program=/bin/fuse-overlayfs"
time="2020-08-06T09:50:01+02:00" level=debug msg="backingFs=xfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=false"
time="2020-08-06T09:50:01+02:00" level=debug msg="Initializing event backend file"
time="2020-08-06T09:50:01+02:00" level=warning msg="Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument"
time="2020-08-06T09:50:01+02:00" level=debug msg="using runtime \"/usr/bin/runc\""
time="2020-08-06T09:50:01+02:00" level=warning msg="Error initializing configured OCI runtime crun: no valid executable found for OCI runtime crun: invalid argument"
time="2020-08-06T09:50:01+02:00" level=info msg="Setting parallel job count to 13"
time="2020-08-06T09:50:01+02:00" level=info msg="Created exec session 5db7fea1745b65b5bc67e0d324a18bda2c7c7d611f50be482eb038ea9b4b0e93 in container c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2"
time="2020-08-06T09:50:01+02:00" level=info msg="Going to start container c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2 exec session 5db7fea1745b65b5bc67e0d324a18bda2c7c7d611f50be482eb038ea9b4b0e93 and attach to it"
time="2020-08-06T09:50:01+02:00" level=debug msg="Sending resize events to exec session 5db7fea1745b65b5bc67e0d324a18bda2c7c7d611f50be482eb038ea9b4b0e93"
time="2020-08-06T09:50:01+02:00" level=debug msg="/usr/bin/conmon messages will be logged to syslog"
time="2020-08-06T09:50:01+02:00" level=debug msg="running conmon: /usr/bin/conmon" args="[--api-version 1 -c c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2 -u 5db7fea1745b65b5bc67e0d324a18bda2c7c7d611f50be482eb038ea9b4b0e93 -r /usr/bin/runc -b /home/cadmin/.local/share/containers/storage/overlay-containers/c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2/userdata/5db7fea1745b65b5bc67e0d324a18bda2c7c7d611f50be482eb038ea9b4b0e93 -p /home/cadmin/.local/share/containers/storage/overlay-containers/c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2/userdata/5db7fea1745b65b5bc67e0d324a18bda2c7c7d611f50be482eb038ea9b4b0e93/exec_pid -n postgresql --exit-dir /home/cadmin/.local/share/containers/storage/overlay-containers/c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2/userdata/5db7fea1745b65b5bc67e0d324a18bda2c7c7d611f50be482eb038ea9b4b0e93/exit --socket-dir-path /tmp/run-1002/libpod/tmp/socket -l none --log-level debug --syslog -e --exec-attach --exec-process-spec /home/cadmin/.local/share/containers/storage/overlay-containers/c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2/userdata/5db7fea1745b65b5bc67e0d324a18bda2c7c7d611f50be482eb038ea9b4b0e93/exec-process-395599256]"
time="2020-08-06T09:50:01+02:00" level=warning msg="Failed to add conmon to cgroupfs sandbox cgroup: error creating cgroup for memory: mkdir /sys/fs/cgroup/memory/libpod_parent: permission denied"
time="2020-08-06T09:50:01+02:00" level=debug msg="Attaching to container c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2 exec session 5db7fea1745b65b5bc67e0d324a18bda2c7c7d611f50be482eb038ea9b4b0e93"
time="2020-08-06T09:50:01+02:00" level=debug msg="connecting to socket /tmp/run-1002/libpod/tmp/socket/5db7fea1745b65b5bc67e0d324a18bda2c7c7d611f50be482eb038ea9b4b0e93/attach"
time="2020-08-06T09:50:01+02:00" level=debug msg="Received: 0"
time="2020-08-06T09:50:01+02:00" level=debug msg="Received: -1"
Error: [conmon:d]: exec with attach is waiting for start message from parent
[conmon:d]: exec with attach got start message from parent
time="2020-08-06T09:50:01+02:00" level=error msg="exec failed: container \"c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2\" does not exist"
exec failed: container "c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2" does not exist: OCI runtime error
**Execution on ssh terminal:**
***@***.***~]$ /bin/podman exec --log-level=debug postgresql /usr/bin/psql -U postgres -l
INFO[0000] /bin/podman filtering at log level debug
DEBU[0000] Called exec.PersistentPreRunE(/bin/podman exec --log-level=debug postgresql /usr/bin/psql -U postgres -l)
DEBU[0000] Ignoring libpod.conf EventsLogger setting "/home/cadmin/.config/containers/containers.conf". Use "journald" if you want to change this setting and remove libpod.conf files.
DEBU[0000] Using conmon: "/usr/bin/conmon"
DEBU[0000] Initializing boltdb state at /home/cadmin/.local/share/containers/storage/libpod/bolt_state.db
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /home/cadmin/.local/share/containers/storage
DEBU[0000] Using run root /tmp/run-1002
DEBU[0000] Using static dir /home/cadmin/.local/share/containers/storage/libpod
DEBU[0000] Using tmp dir /tmp/run-1002/libpod/tmp
DEBU[0000] Using volume path /home/cadmin/.local/share/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] Not configuring container store
DEBU[0000] Initializing event backend file
DEBU[0000] using runtime "/usr/bin/runc"
WARN[0000] Error initializing configured OCI runtime crun: no valid executable found for OCI runtime crun: invalid argument
WARN[0000] Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument
INFO[0000] Setting parallel job count to 13
INFO[0000] /bin/podman filtering at log level debug
DEBU[0000] Called exec.PersistentPreRunE(/bin/podman exec --log-level=debug postgresql /usr/bin/psql -U postgres -l)
DEBU[0000] Ignoring libpod.conf EventsLogger setting "/home/cadmin/.config/containers/containers.conf". Use "journald" if you want to change this setting and remove libpod.conf files.
DEBU[0000] Using conmon: "/usr/bin/conmon"
DEBU[0000] Initializing boltdb state at /home/cadmin/.local/share/containers/storage/libpod/bolt_state.db
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /home/cadmin/.local/share/containers/storage
DEBU[0000] Using run root /tmp/run-1002
DEBU[0000] Using static dir /home/cadmin/.local/share/containers/storage/libpod
DEBU[0000] Using tmp dir /tmp/run-1002/libpod/tmp
DEBU[0000] Using volume path /home/cadmin/.local/share/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] overlay: mount_program=/bin/fuse-overlayfs
DEBU[0000] overlay: mount_program=/bin/fuse-overlayfs
DEBU[0000] backingFs=xfs, projectQuotaSupported=false, useNativeDiff=false, usingMetacopy=false
DEBU[0000] Initializing event backend file
DEBU[0000] using runtime "/usr/bin/runc"
WARN[0000] Error initializing configured OCI runtime crun: no valid executable found for OCI runtime crun: invalid argument
WARN[0000] Error initializing configured OCI runtime kata: no valid executable found for OCI runtime kata: invalid argument
INFO[0000] Setting parallel job count to 13
INFO[0000] Created exec session 3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0 in container c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2
INFO[0000] Going to start container c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2 exec session 3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0 and attach to it
DEBU[0000] Sending resize events to exec session 3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0
DEBU[0000] /usr/bin/conmon messages will be logged to syslog
DEBU[0000] running conmon: /usr/bin/conmon args="[--api-version 1 -c c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2 -u 3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0 -r /usr/bin/runc -b /home/cadmin/.local/share/containers/storage/overlay-containers/c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2/userdata/3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0 -p /home/cadmin/.local/share/containers/storage/overlay-containers/c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2/userdata/3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0/exec_pid -n postgresql --exit-dir /home/cadmin/.local/share/containers/storage/overlay-containers/c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2/userdata/3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0/exit --socket-dir-path /tmp/run-1002/libpod/tmp/socket -l none --log-level debug --syslog -e --exec-attach --exec-process-spec /home/cadmin/.local/share/containers/storage/overlay-containers/c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2/userdata/3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0/exec-process-989882471]"
WARN[0000] Failed to add conmon to cgroupfs sandbox cgroup: error creating cgroup for blkio: mkdir /sys/fs/cgroup/blkio/libpod_parent: permission denied
DEBU[0000] Attaching to container c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2 exec session 3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0
DEBU[0000] connecting to socket /tmp/run-1002/libpod/tmp/socket/3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0/attach
DEBU[0000] Received: 0
DEBU[0000] Received: 87916
DEBU[0000] Successfully started exec session 3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0 in container c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2
[conmon:d]: exec with attach is waiting for start message from parent
[conmon:d]: exec with attach got start message from parent
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
------------------+----------+----------+------------+------------+------------------------------------
<shortened output >
(6 rows)
DEBU[0000] Container c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2 exec session 3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0 completed with exit code 0
INFO[0000] Removing container c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2 exec session 3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0
DEBU[0000] Successfully removed container c08dc8d38d4cdae20d762b3f142ea81d10f54b2ca04308e930f5bbd66ca712b2 exec session 3e506c058f1261526a50cfd20f13051143ffac2c363e72951b1147f45b5cedf0
DEBU[0000] Called exec.PersistentPostRunE(/bin/podman exec --log-level=debug postgresql /usr/bin/psql -U postgres -l)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#7242>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AB3AOCD6VDX55JRCKV7RZGTR7JPBNANCNFSM4PWJ64YQ>
.
|
|
Hello, |
|
This sounds like a good addition to the known issue @mheon |
|
I do believe it's already there. We do have a discoverability problem around the known issues page - I have never once seen it among our most visited pages. |
|
Yeah, helping the discoverability for that page is on my back burner list of todo's, I really need to pop that up a stack or two in the queue. |
|
Should we add a link to podman.io? Should we put a link into the podman man page? |
|
podman.io sounds like a good place |
|
The problem is fixed for me. So im closing the issue for now. |
ghost
mentioned this issue
github-actions
bot
added
the
locked - please file new issue/PR
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
/kind bug
Description
Podman not working with cron
Steps to reproduce the issue:
1.Edit as cadmin User crontab -e
50 * * * * /opt/scripts/backup/postgresql_backup.sh >> /opt/logs/backup/postgresql_backup.log 2>&1
#!/bin/bash
whoami
/bin/podman exec --log-level=debug postgresql /usr/bin/psql -U postgres -l
Describe the results you received:
podman command is working fine on a normal terminal. I got the result of the psql-command. See below.
I also tried setting the same env vars from my local terminal into the bash script. Problem still exists.
I tried setting the path env var in the script.
Describe the results you expected:
Podman cronjob is succeeding
Versions
[cadmin@serverchanged~]$ cat /etc/redhat-release
CentOS Linux release 8.2.2004 (Core)
[cadmin@serverchanged ~]$ podman version
Version: 2.0.4
API Version: 1
Go Version: go1.13.4
Built: Thu Jan 1 01:00:00 1970
OS/Arch: linux/amd64
The text was updated successfully, but these errors were encountered: