Container does not create source directory on the host for Binds

[linggao]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description
When starting a container using the docker compatible REST APIs, error occurred complaining that the source directory does not exist for the Binds specified in the HostConfig.
If running the same script on a docker engine, docker automatically creates the source directory if it does not exists.

Steps to reproduce the issue:

1. Run this script:

#!/bin/bash

# create container
echo -e "\ncreating container test..."

read -d '' sdef <<EOF
{
  "Image": "alpine",
  "Cmd": [
      "sleep", "1000"
  ],
  "HostConfig": {
    "Binds": ["/tmp/testdata3:/tmp/data3:ro"]
  }
}
EOF
echo "$sdef" | curl -sSLw "%{http_code}" -X POST -H "Content-Type: application/json" --unix-socket /var/run/podman/podman.sock --data @-  http://localhost/containers/create?name=test

echo -e "\n\n start container test"
curl -sSLw "%{http_code}" -X POST -H "Content-Type: application/json" --unix-socket /var/run/podman/podman.sock  http://localhost/containers/test/start

Describe the results you received:
creating container test...
{"cause":"statfs /tmp/testdata3: no such file or directory","message":"container create: statfs /tmp/testdata3: no such file or directory","response":500}
500

Describe the results you expected:
Directory /tmp/testdata3 should be automatically created on the host.

Additional information you deem important (e.g. issue happens only occasionally):
Command podman run --name test1 -v /tmp/mydata:/tmp/data -d alpine sleep 1000 gives the same error

Output of podman version:

Version:      3.1.0-dev
API Version:  3.0.0
Go Version:   go1.14.12
Git Commit:   1702cbc6917f431bcc65d6c5bdc6fcf99231977a-dirty
Built:        Wed Feb 24 09:13:30 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.19.6
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: Unknown
    path: /usr/local/libexec/podman/conmon
    version: 'conmon version 2.0.27-dev, commit: 7310bf13319ee8ed50799b202509bedc27b36cf8'
  cpus: 2
  distribution:
    distribution: '"rhel"'
    version: "8.3"
  eventLogger: file
  hostname: lingvs4.dev.edge-fabric.com
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 4.18.0-240.15.1.el8_3.x86_64
  linkmode: dynamic
  memFree: 3221221376
  memTotal: 8342470656
  ociRuntime:
    name: runc
    package: Unknown
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc93+dev
      commit: f245a1d1edbf545549e5a16106cf1aec356a3c7d
      spec: 1.0.2-dev
      go: go1.14.12
      libseccomp: 2.4.3
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_MKNOD,CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    selinuxEnabled: true
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 2146758656
  swapTotal: 2146758656
  uptime: 154h 7m 22.62s (Approximately 6.42 days)
registries:
  search:
  - registry.access.redhat.com
  - registry.redhat.io
  - docker.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 2
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 9
  runRoot: /var/run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.0.0
  Built: 1614179610
  BuiltTime: Wed Feb 24 09:13:30 2021
  GitCommit: 1702cbc6917f431bcc65d6c5bdc6fcf99231977a-dirty
  GoVersion: go1.14.12
  OsArch: linux/amd64
  Version: 3.1.0-dev

Package info (e.g. output of rpm -q podman or apt list podman):

podman is build locally with the the latest code. podman rpm is not installed. 

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

[mheon]

@rhatdan @baude We decided not to do this for podman run because we felt it was most likely user error, but this is the Docker-compat API, so an argument could be made that we should do it here?

[rhatdan]

@linggao Could you explain why you would want this feature. My argument against it at least from the podman client side, it is enables users to accidentally create unintended directories, when they have a typo in their command. I don't believe people would expect

mount -o bind /tmp/testdata3 /tmp/data3

To automatically create /tmp/testdata3 directory, which is why i don't want podman do do it either.

podman -v /tmp/testdata3:/tmp/data3 ...

[dabooz]

@rhatdan At the risk of sounding overly pejorative, when it comes to the compatibility API, if docker supports it then your API should too. I think that has to be a foundation principle for supporting the API. I do expect there will be cases where a reasonable exception can be made (I can cite an example if you want it). But, I don't think this is one of them. Docker seems to have already decided (based on how the API behaves) that this scenario is not a user error. I agree that if the behavior of the API were being designed right now, we could argue for your behavior, and I might even agree with you. But that's all a mute point now.

[rhatdan]

Ok I will succumb for the Docker Compatibility API. Bug for Bug compatibility.

[gogodave]

I don't know if this was ever working, but I have tested this on 4.4.1 and 4.5.1, and it does not work as expected. I get this:

podman run --rm -v /tmp/$(uuidgen):/uuid alpine:latest echo works
Error: statfs /tmp/39F1DF6B-2117-4FFC-9D17-4634A7702223: no such file or directory

Can we reopen this?
Thanks

[rhatdan]

Podman is not using the docker compatibility layer. You need to use something like docker-py or docker-compose to use compat.
Podman client NEVER creates source for volumes.

[domdomegg]

(duplicate of #6234)