"podman cp" handles "/dev/stdout" as "-" (docker incompatibility)

[eriksjolund]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

The command
docker cp foobar:/root /dev/stdout
fails.
The same command with Podman does not fail, instead Podman writes a tar stream to stdout.
To match the Docker behaviour, Podman should fail too.

Steps to reproduce the issue:

Running with Podman

esjolund@laptop:~$ podman --version
podman version 3.0.0
esjolund@laptop:~$ podman create --name foobar docker.io/library/alpine:latest
6a4ea49f6bbc06572bbb117d6afeec54a1ab90af8645248ed0bfc462c06a0f72
esjolund@laptop:~$ rm -f /tmp/cmd.stdout
esjolund@laptop:~$ rm -f /tmp/cmd.stderr
esjolund@laptop:~$ podman cp foobar:/root /dev/stdout > /tmp/cmd.stdout 2> /tmp/cmd.stderr
esjolund@laptop:~$ echo $?
0
esjolund@laptop:~$ ls -l /tmp/cmd.std*
-rw-rw-r-- 1 esjolund esjolund    0 feb 14 18:55 /tmp/cmd.stderr
-rw-rw-r-- 1 esjolund esjolund 1536 feb 14 18:55 /tmp/cmd.stdout
esjolund@laptop:~$ cat /tmp/cmd.stdout | tar tvf -
drwx------ root/root         0 2021-01-28 22:51 root
esjolund@laptop:~$ rm -f /tmp/cmd.stdout
esjolund@laptop:~$ rm -f /tmp/cmd.stderr
esjolund@laptop:~$ podman cp foobar:/root /dev/stdout
root0000700000000000000000000000000014004630720011742 5ustar00rootroot00000000000000esjolund@laptop:~$ echo $?
0
esjolund@laptop:~$ 

Running with Docker

esjolund@laptop:~$ docker --version
Docker version 20.10.3, build 48d30b5
esjolund@laptop:~$ docker create --name foobar docker.io/library/alpine:latest
Unable to find image 'alpine:latest' locally
latest: Pulling from library/alpine
4c0d98bf9879: Pull complete 
Digest: sha256:08d6ca16c60fe7490c03d10dc339d9fd8ea67c6466dea8d558526b1330a85930
Status: Downloaded newer image for alpine:latest
18ab1740fb3a784596ba47088d773894e111db872d5af7db00eabe202bfb44d9
esjolund@laptop:~$ rm -f /tmp/cmd.stdout
esjolund@laptop:~$ rm -f /tmp/cmd.stderr
esjolund@laptop:~$ docker cp foobar:/root /dev/stdout > /tmp/cmd.stdout 2> /tmp/cmd.stderr
esjolund@laptop:~$ echo $?
1
esjolund@laptop:~$ ls -l /tmp/cmd*
-rw-rw-r-- 1 esjolund esjolund 22 feb 14 18:49 /tmp/cmd.stderr
-rw-rw-r-- 1 esjolund esjolund  0 feb 14 18:49 /tmp/cmd.stdout
esjolund@laptop:~$ cat /tmp/cmd.stderr 
cannot copy directory
esjolund@laptop:~$ rm /tmp/cmd.std*
esjolund@laptop:~$ docker cp foobar:/root /dev/stdout
invalid output path: "/dev/stdout" must be a directory or a regular file: got a device
esjolund@laptop:~$ echo $?
1
esjolund@laptop:~$ 

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

podman version 3.0.0

Output of podman info --debug | grep -v hostname::

host:
  arch: amd64
  buildahVersion: 1.19.2
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: 'conmon: /usr/libexec/podman/conmon'
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.26, commit: '
  cpus: 4
  distribution:
    distribution: ubuntu
    version: "20.04"
  eventLogger: journald
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.4.0-65-generic
  linkmode: dynamic
  memFree: 9832243200
  memTotal: 16666775552
  ociRuntime:
    name: crun
    package: 'crun: /usr/bin/crun'
    path: /usr/bin/crun
    version: |-
      crun version 0.17.6-58ef-dirty
      commit: fd582c529489c0738e7039cbc036781d1d039014
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    selinuxEnabled: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: 'slirp4netns: /usr/bin/slirp4netns'
    version: |-
      slirp4netns version 1.1.8
      commit: unknown
      libslirp: 4.3.1-git
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.4.3
  swapFree: 2147479552
  swapTotal: 2147479552
  uptime: 70h 38m 28.11s (Approximately 2.92 days)
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /home/esjolund/.config/containers/storage.conf
  containerStore:
    number: 5
    paused: 0
    running: 1
    stopped: 4
  graphDriverName: vfs
  graphOptions: {}
  graphRoot: /home/esjolund/.local/share/containers/storage
  graphStatus: {}
  imageStore:
    number: 23
  runRoot: /run/user/1000/containers
  volumePath: /home/esjolund/.local/share/containers/storage/volumes
version:
  APIVersion: 3.0.0
  Built: 0
  BuiltTime: Thu Jan  1 01:00:00 1970
  GitCommit: ""
  GoVersion: go1.15.2
  OsArch: linux/amd64
  Version: 3.0.0

Package info (e.g. output of rpm -q podman or apt list podman):

Listing... Done
podman/unknown,now 100:3.0.0-1 amd64 [installed]
podman/unknown 100:3.0.0-1 arm64
podman/unknown 100:3.0.0-1 armhf
podman/unknown 100:3.0.0-1 s390x

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

[vrothberg]

Thanks, @eriksjolund!

I can tackle the issue as I caused the incompatibility :^) The philosophical question whether it's a bug or feature but Docker compatibility is the goal.

[vrothberg]

The more I think about it, the greater the feelings that this is a bug in Docker. - and /dev/stdout are essentially the same.

@eriksjolund is there a specific reason you consider it a bug? Some tools that may depend on this specific behavior?

[eriksjolund]

- is used more as a command-line option. - does not have a meaning as path.
But /dev/stdout is not used as a verbatim string. Instead it gives the impression that it is handled as a path in the file system.

I tried out a few examples:

1. podman cp foobar:/root /dev/stdout
2. cd /dev && podman cp foobar:/root stdout
3. dereferencing /dev/stdout and instead copying to /proc/self/fd/1

Nr 1 and nr 2 succeeded and gave the same result.
Nr 3 failed.

(I was logged in over ssh to this computer)

esjolund@linux:~$ podman cp foobar:/root /dev/stdout > /tmp/o1.txt 2> /tmp/e1.txt
esjolund@linux:~$ cd /dev/
esjolund@linux:/dev$ podman cp foobar:/root stdout > /tmp/o2.txt 2> /tmp/e2.txt
esjolund@linux:/dev$ diff -u /tmp/o1.txt  /tmp/o2.txt 
esjolund@linux:/dev$ tar tvf /tmp/o1.txt 
drwx------ root/root         0 2021-02-17 16:07 root
esjolund@linux:/dev$ ls -l /dev/stdout 
lrwxrwxrwx 1 root root 15 feb 20 02:00 /dev/stdout -> /proc/self/fd/1
esjolund@linux:/dev$ podman cp foobar:/root /proc/self/fd/1 > /tmp/o3.txt 2> /tmp/e3.txt
esjolund@linux:/dev$ ls -l /tmp/[oe][123].txt
-rw-rw-r-- 1 esjolund esjolund    0 feb 20 21:46 /tmp/e1.txt
-rw-rw-r-- 1 esjolund esjolund    0 feb 20 21:47 /tmp/e2.txt
-rw-rw-r-- 1 esjolund esjolund 1435 feb 20 21:49 /tmp/e3.txt
-rw-rw-r-- 1 esjolund esjolund 1536 feb 20 21:46 /tmp/o1.txt
-rw-rw-r-- 1 esjolund esjolund 1536 feb 20 21:47 /tmp/o2.txt
-rw-rw-r-- 1 esjolund esjolund    0 feb 20 21:49 /tmp/o3.txt
esjolund@linux:/dev$ cat /tmp/e3.txt 
Error: 1 error occurred:
	* error copying to host: error during bulk transfer for copier.request{Request:"PUT", Root:"/", preservedRoot:"/proc/self/fd", rootPrefix:"/proc/self/fd", Directory:"/", preservedDirectory:"/proc/self/fd", Globs:[]string{}, preservedGlobs:[]string{}, StatOptions:copier.StatOptions{CheckForArchives:false, Excludes:[]string(nil)}, GetOptions:copier.GetOptions{UIDMap:[]idtools.IDMap(nil), GIDMap:[]idtools.IDMap(nil), Excludes:[]string(nil), ExpandArchives:false, ChownDirs:(*idtools.IDPair)(nil), ChmodDirs:(*os.FileMode)(nil), ChownFiles:(*idtools.IDPair)(nil), ChmodFiles:(*os.FileMode)(nil), StripSetuidBit:false, StripSetgidBit:false, StripStickyBit:false, StripXattrs:false, KeepDirectoryNames:false, Rename:map[string]string(nil)}, PutOptions:copier.PutOptions{UIDMap:[]idtools.IDMap(nil), GIDMap:[]idtools.IDMap(nil), DefaultDirOwner:(*idtools.IDPair)(nil), DefaultDirMode:(*os.FileMode)(nil), ChownDirs:(*idtools.IDPair)(0xc0000f5790), ChmodDirs:(*os.FileMode)(nil), ChownFiles:(*idtools.IDPair)(0xc0000f57a0), ChmodFiles:(*os.FileMode)(nil), StripXattrs:false, IgnoreXattrErrors:false, IgnoreDevices:false, NoOverwriteDirNonDir:false, Rename:map[string]string(nil)}, MkdirOptions:copier.MkdirOptions{UIDMap:[]idtools.IDMap(nil), GIDMap:[]idtools.IDMap(nil), ChownNew:(*idtools.IDPair)(nil), ChmodNew:(*os.FileMode)(nil)}}: copier: put: error creating "/root": mkdir /root: no such file or directory
esjolund@linux:/dev$ 

I think adding support for /dev/stdout adds unnecessary complexity and ambiguity to the podman cp command.
As this is more or less a user-interface question, I think choosing the same as Docker makes most sense so that users will have less to learn when they migrate to Podman.

[vrothberg]

Thanks for coming back, @eriksjolund.

Nr 1 and nr 2 succeeded and gave the same result.
Nr 3 failed.

That's a convincing argument to not treat /dev/stdout specially. I'll tackle it today.