ignore_chown_errors is ignored in global storage.conf

[plopresti]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

ignore_chown_errors = "true" does not work if I put it in /etc/containers/storage.conf. It does work if I put it in ~/.config/containers/storage.conf.

Steps to reproduce the issue:

$ rm ~/.config/containers/storage.conf
$ cat /etc/containers/storage.conf
[storage]
  driver = "overlay"
  [storage.options]
    mount_program = "/bin/fuse-overlayfs"
    [storage.options.overlay]
      ignore_chown_errors = "true"
$ podman info | grep ignore_chown_errors
$ cp /etc/containers/storage.conf ~/.config/containers/.
$ podman info | grep ignore_chown_errors
    overlay.ignore_chown_errors: "true"

Describe the results you received:
"pm info | grep ignore_chown_errors" shows nothing even though I set ignore_chown_errors in /etc/containers/storage.conf

Describe the results you expected:
I expect the ignore_chown_errors setting in /etc/containers/storage.conf to be honored

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Version:      2.0.5
API Version:  1
Go Version:   go1.13.14
Built:        Fri Aug 28 11:06:26 2020
OS/Arch:      linux/amd64

Output of podman info --debug:

$ pm info --debug
host:
  arch: amd64
  buildahVersion: 1.15.1
  cgroupVersion: v1
  conmon:
    package: conmon-2.0.20-1.el7.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.20, commit: 2eef3028b91df012a014c0b8ceb3202b3d2222bb'
  cpus: 48
  distribution:
    distribution: '"rhel"'
    version: "7.8"
  eventLogger: file
  hostname: lex3.kla-tencor.com
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 100
      size: 1
    uidmap:
    - container_id: 0
      host_id: 39327
      size: 1
  kernel: 3.10.0-1127.19.1.el7.x86_64
  linkmode: dynamic
  memFree: 396886724608
  memTotal: 404129320960
  ociRuntime:
    name: runc
    package: runc-1.0.0-103.dev.el7.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc10
      commit: 74adef73170b04a36df2fe6f3233761640e59625
      spec: 1.0.1-dev
  os: linux
  remoteSocket:
    path: /run/user/39327/podman/podman.sock
  rootless: true
  slirp4netns:
    executable: /bin/slirp4netns
    package: slirp4netns-1.1.4-2.el7.x86_64
    version: |-
      slirp4netns version 1.1.4
      commit: b66ffa8e262507e37fca689822d23430f3357fe8
      libslirp: 4.3.1
      SLIRP_CONFIG_VERSION_MAX: 3
  swapFree: 34359734272
  swapTotal: 34359734272
  uptime: 23m 54.37s
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - docker.io
store:
  configFile: /home/ploprest/.config/containers/storage.conf
  containerStore:
    number: 3
    paused: 0
    running: 0
    stopped: 3
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /bin/fuse-overlayfs
      Package: fuse-overlayfs-1.1.2-1.el7.x86_64
      Version: |-
        fusermount3 version: 3.6.1
        fuse-overlayfs: version 1.1.0
        FUSE library version 3.6.1
        using FUSE kernel interface version 7.29
  graphRoot: /scratch/patl/podman
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 17
  runRoot: /run/user/39327/containers
  volumePath: /scratch/patl/podman/volumes
version:
  APIVersion: 1
  Built: 1598637986
  BuiltTime: Fri Aug 28 11:06:26 2020
  GitCommit: ""
  GoVersion: go1.13.14
  OsArch: linux/amd64
  Version: 2.0.5

Package info (e.g. output of rpm -q podman or apt list podman):

podman-2.0.5-2.el7.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

[mheon]

@nalind @rhatdan PTAL - I was under the impression that we based rootless storage configuration on global storage configuration, but I could be mistaken.

[nalind]

If I'm reading it right, it's currently an either-or thing. I could see us wanting to change that, and wiring in a /usr/share/containers/storage.conf while we're in there.

[plopresti]

@mheon @nalind I do not understand your replies. Are you saying my bug report is incorrect?

Simply put, I want to enable ignore_chown_errors globally for all users. Is there a way to do this?

[mheon]

@plopresti My understanding was that the global storage configuration file was taken into account when generating per-user storage configuration, which would mean I would expect your adding configuration to that file would work.

@nalind has indicated that is not presently the case, though, so there is presently no way to set global storage configuration options for rootless Podman at present.

[mheon]

It sounds like something we should allow, though - your use-case of globally setting ignore-chown-errors is 100% valid.

[plopresti]

@mheon This version of podman does not generate any per-user storage configuration. (If you start with no ~/.config/containers/storage.conf, running podman does not create it.)

This is kind of how I noticed... My own storage.conf was generated the first time I ran podman a while ago. So I had instructions for people to edit their per-user storage.conf after running "podman" the first time. But they cannot follow those instructions because it is no longer created. So I figured maybe I would just add ignore_chown_errors to the global settings file and be done with it. Then I discovered that does not work, so I filed this bug.

It seems like podman is a little confused/confusing on how it handles config files in general (?) Or is it just for this setting?

[mheon]

Creating config files on first run had its own set of problems (#7509 is an example) so we decided to disable it. You should still be able to create the file yourself with minimal contents (just the basic structure and ignore_chown_errors set, everything else commented out) and we will still read and use it.

[rhatdan]

This should work.

# grep ignore_chown /etc/containers/storage.conf
# ignore_chown_errors can be set to allow a non privileged user running with
ignore_chown_errors = "true"
# podman info | grep ignore
    overlay.ignore_chown_errors: "true"
$ rm -f ~/.config/containers/storage.conf
$ podman info | grep ignore
$

So it looks like this field is being ignored by rootless containers, This is definitely a bug.

[rhatdan]

We need to get this fixed ASAP. It is needed for HPC Environmnets.

[plopresti]

@rhatdan I would expect the storage configuration to be implemented like this:

1. Assign defaults to all settings
2. Read /etc/containers/storage.conf, applying any settings therein to override (1)
3. Read ~/.config/containers/storage.conf, applying any settings therein to override (1) and (2)
4. Parse the command line, applying any settings therein to override (1), (2), and (3)
5. Run with the final settings

Obviously, this is not how it works... Because if it were, this bug itself would be "impossible by construction".

So my question is, how DOES it work; and, more importantly, where is it documented?

[rhatdan]

Podman 2.0.7 should have this fix.