Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

generated systemd unit files need to pull in network-online.target #10655

Closed
srd424 opened this issue Jun 11, 2021 · 7 comments · Fixed by #10962
Closed

generated systemd unit files need to pull in network-online.target #10655

srd424 opened this issue Jun 11, 2021 · 7 comments · Fixed by #10962
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@srd424
Copy link

srd424 commented Jun 11, 2021

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Generated systemd unit files need to pull in network-online.target; at the moment they have After=network-online.target, but as per the systemd documentation, they should have Wants=network-online.target as well.

In my particular case, I've seen podman starting containers before the host has finished using DHCP to configure the main interface, which in turns means the host resolv.conf has not been correctly populated and is therefore not correctly configured in the containers either.

Steps to reproduce the issue:

(my particular instance of this problem is a race condition so hard to create a simple / scriptable reproducer; hopefully examination of systemd docs and podman generated unit files make this "obvious to inspection.")

Describe the results you received:

Started containers cannot always access DNS.

Describe the results you expected:

Started containers should reliably be able to access the network.

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Version:      3.1.2
API Version:  3.1.2
Go Version:   go1.15.2
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.20.1
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: 'conmon: /usr/libexec/podman/conmon'
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.27, commit: '
  cpus: 4
  distribution:
    distribution: ubuntu
    version: "20.04"
  eventLogger: journald
  hostname: asr-ocihost
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.4.0-74-generic
  linkmode: dynamic
  memFree: 2218684416
  memTotal: 3826343936
  ociRuntime:
    name: crun
    package: 'crun: /usr/bin/crun'
    path: /usr/bin/crun
    version: |-
      crun version 0.19.1.3-9b83-dirty
      commit: 33851ada2cc9bf3945915565bf3c2df97facb92c
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    selinuxEnabled: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 35m 41.93s
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.imagestore: /vol/ocistore
    overlay.mountopt: nodev
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 16
  runRoot: /var/run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.1.2
  Built: 0
  BuiltTime: Thu Jan  1 01:00:00 1970
  GitCommit: ""
  GoVersion: go1.15.2
  OsArch: linux/amd64
  Version: 3.1.2

Package info (e.g. output of rpm -q podman or apt list podman):

Listing... Done
podman/unknown,now 100:3.1.2-1 amd64 [installed]
podman/unknown 100:3.1.2-1 arm64
podman/unknown 100:3.1.2-1 armhf
podman/unknown 100:3.1.2-1 s390x

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

3.2 is not yet in the apt repos it seems; inspection of the source on github suggests the relevant code has not recently changed.

Additional environment details (AWS, VirtualBox, physical, etc.):

podman containers are nested within a systemd container environment (probably best not to ask!)

@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Jun 11, 2021
@Luap99
Copy link
Member

Luap99 commented Jun 11, 2021

This sounds good to me.
@srd424 Interested in opening a PR to add this?

@rhatdan
Copy link
Member

rhatdan commented Jun 11, 2021

@vrothberg FYI

@vrothberg
Copy link
Member

Sounds good to me. Indeed the man pages suggest that Wants= would address the issue.

@cdoern
Copy link
Contributor

cdoern commented Jun 18, 2021

@vrothberg would this be as simple as changing Wants=network.target to Wants=network.online.target in common.go? If so I can open up a PR.

@Luap99
Copy link
Member

Luap99 commented Jun 18, 2021

@cdoern Yes

@vrothberg
Copy link
Member

@cdoern, the unit tests will require some massaging as well. You can run them in the project root with go test -v ./pkg/systemd/generate.

vrothberg added a commit to vrothberg/libpod that referenced this issue Jul 16, 2021
Require the network to be online in all (generated) systemd units to
make sure that containers and Podman run only after the network has been
fully configured.

Fixes: containers#10655
Signed-off-by: Valentin Rothberg <[email protected]>
@dustymabe
Copy link
Contributor

It's worth noting that depending on network-online.target won't have the desired effect for any systemd user units (i.e. the ones you put in ~/.config/systemd/user).

See systemd/systemd#3312 (comment)

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants