podman: fails when underlying filesystem does not support security labels (EXT4_FS_SECURITY)

[flyn-org]

/kind bug

Description

I am working with podman on OpenWrt, a light-weight distribution that aims to support network devices. The default OpenWrt kernel does not contain support for EXT4 security labels (EXT4_FS_SECURITY), and this seems to cause podman to fail under some circumstances. I am not sure if security labels are strictly required by podman, or if they merely increase the security surrounding containers (i.e., is it reasonable to run podman without security labels?).

Steps to reproduce the issue:

1. Install podman and sudo on OpenWrt.

2. Build a container image named "foo".

3. Run sudo -u user sh

4. Run podman run --userns=auto foo

Describe the results you received:

Without support for security labels, I get:

Error: error creating container storage: error creating an ID-mapped copy of layer "[hash]": exit status 1: error during chown: storage-chown-by-maps: lgetxattr bin: operation not supported

Describe the results you expected:

I think podman should be able to run without security labels. Obviously, it would not benefit from things like SELinux or capabilities if this were the case.

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

# podman version
Version:      2.2.1
API Version:  2.1.0
Go Version:   go1.16
Built:        Wed Dec 31 18:00:00 1969
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.18.0
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
    package: Unknown
    path: /usr/bin/conmon
    version: 'conmon version 2.0.26, commit: '
  cpus: 1
  distribution:
    distribution: '"openwrt"'
    version: snapshot
  eventLogger: file
  hostname: host
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.4.102
  linkmode: dynamic
  memFree: 1906167808
  memTotal: 2074664960
  ociRuntime:
    name: /usr/sbin/runc
    package: Unknown
    path: /usr/sbin/runc
    version: |-
      runc version 1.0.0-rc93
      commit: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
      spec: 1.0.2-dev
      go: go1.16
      libseccomp: 2.5.1
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  rootless: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 9m 8.7s
registries:
  search:
  - docker.io
  - registry.fedoraproject.org
  - registry.access.redhat.com
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: tmpfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 0
  runRoot: /var/run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 2.1.0
  Built: 0
  BuiltTime: Wed Dec 31 18:00:00 1969
  GitCommit: ""
  GoVersion: go1.16
  OsArch: linux/amd64
  Version: 2.2.1

Package info (e.g. output of rpm -q podman or apt list podman):

# opkg list-installed | grep podman
podman - 2.2.1-2

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

I visually inspected the source code in master.

Additional environment details (AWS, VirtualBox, physical, etc.):

vendor/github.com/containers/storage/drivers/chown_unix.go contains this:

cap, err := system.Lgetxattr(path, "security.capability")
if err != nil && err != system.ErrNotSupportedPlatform {
        return fmt.Errorf("%s: %v", os.Args[0], err)
}

[...]

if cap != nil {
        if err := system.Lsetxattr(path, "security.capability", cap, 0); err != nil {
                return fmt.Errorf("%s: %v", os.Args[0], err)
        }
}

and chown.go makes use of this with:

if err := pwalk.Walk(".", chown); err != nil {
        fmt.Fprintf(os.Stderr, "error during chown: %v", err)
        os.Exit(1)
}

Would it be better to catch the idea that "security.capability" is unsupported and thus avoid the later Lsetxattr? The current approach is to error out if Lgetxattr failes.

[mheon]

This isn't SELinux. This is a lack of system support for, at the minimum, security-related extended attributes in the filesystem (possibly all extended attributes). Podman is attempting to preserve extended attributes of the files we're chowning to match the IDs of the user namespace. This shouldn't be fatal, though. It seems like there's already an attempt to ignore getting xattrs if it fails with ENOTSUP (at least I'm assuming that's what system.ErrNotSupportedPlatform is checking for?) but it doesn't look like the check is working. @nalind PTAL

[nalind]

Yes, it looks like we missed a caller when github.com/containers/storage/pkg/system.Lgetxattr() switched to returning wrapped errors, so the error it returns isn't being checked properly. Yup, that's a place where we're not checking for EOPNOTSUPP.