podman-3.2.3 import result image is empty

[babykart]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

podman-3.2.3 import result image is empty

Steps to reproduce the issue:

wget https://mirror.init7.net/gentoo/releases/amd64/autobuilds/current-stage3-amd64-hardened+nomultilib/stage3-amd64-hardened+nomultilib-20210630T214504Z.tar.xz

xzcat stage3-amd64-hardened+nomultilib-20210630T214504Z.tar.xz | bzip2 | podman import - kubler-gentoo/stage3-amd64-hardened-nomultilib:20210630T214504Z

Describe the results you received:

Getting image source signatures
Copying blob e3b0c44298fc done
Copying config 4c17b62dd9 done
Writing manifest to image destination
Storing signatures
sha256:ad8d30dbb5cc8c1c93d668cc6688f0301179270a3e19fc58c1c4f77a55604b31

podman images
localhost/kubler-gentoo/stage3-amd64-hardened-nomultilib  latest            ad8d30dbb5cc  2 minutes ago      1.09 kB

Describe the results you expected:

Getting image source signatures
Copying blob ceeec99a2999 done
Copying config e08265be09 done
Writing manifest to image destination
Storing signatures
e08265be09ba19106937104d11f71a3927d1b031584b4ed0522883fcf8232b9d

podman images
localhost/kubler-gentoo/stage3-amd64-hardened-nomultilib  latest            e08265be09ba  4 minutes ago       905 MB

Additional information you deem important (e.g. issue happens only occasionally):

It appears to be relative to the size of the archive because if I try a smaller archive, it works perfectly with 3.2.X

localhost/kubler-gentoo/portage                           latest            174eb8a7c4a7  2 hours ago     372 MB

Output of podman version:

Version:      3.2.3
API Version:  3.2.3
Go Version:   go1.16.6
Git Commit:   1e6fd46e91b21342f9454cf8105a92b90e398c52
Built:        Tue Jul 20 16:21:49 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.21.3
  cgroupControllers:
  - cpuset
  - cpu
  - cpuacct
  - blkio
  - memory
  - devices
  - freezer
  - net_cls
  - perf_event
  - net_prio
  - hugetlb
  - pids
  - rdma
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: app-emulation/conmon-2.0.29
    path: /usr/bin/conmon
    version: 'conmon version 2.0.29, commit: 7e6de6678f6ed8a18661e1d5721b81ccee293b9b'
  cpus: 8
  distribution:
    distribution: gentoo
    version: unknown
  eventLogger: journald
  hostname: babykart
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.10.49-gentoo-r1
  linkmode: dynamic
  memFree: 3080859648
  memTotal: 16786358272
  ociRuntime:
    name: crun
    package: app-emulation/crun-0.20.1
    path: /usr/bin/crun
    version: |-
      crun version 0.20.1
      commit: 38271d1c8d9641a2cdc70acfa3dcb6996d124b3d
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_MKNOD,CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 142h 48m 23.18s (Approximately 5.92 days)
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /vol/containers/storage
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 65
  runRoot: /run/containers/storage
  volumePath: /vol/containers/storage/volumes
version:
  APIVersion: 3.2.3
  Built: 1626790909
  BuiltTime: Tue Jul 20 16:21:49 2021
  GitCommit: 1e6fd46e91b21342f9454cf8105a92b90e398c52
  GoVersion: go1.16.6
  OsArch: linux/amd64
  Version: 3.2.3

Package info (e.g. output of eix app-emulation/podman):

[I] app-emulation/podman
     Available versions:  (~)3.1.2^t (~)3.2.1^t (~)3.2.3^t[1] {apparmor btrfs +fuse +rootless selinux}
     Installed versions:  3.2.3^t[1](04:23:28 PM 07/20/2021)(btrfs rootless -apparmor -fuse -selinux)
     Homepage:            https://github.com/containers/podman/
     Description:         Library and podman tool for running OCI-based containers in Pods

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

Physical node.

[vrothberg]

Thanks, @babykart! I will take a look at this.

[hoshsadiq]

Got the same issue. Both for rootless and rootfull.

[vrothberg]

Interesting. From reproducing locally, it seems that problem relates to when reading the archive from stdin. Importing the file directly podman import archive.tar works.

[hoshsadiq]

That would make sense. The only times I've used it is by reading from stdin!

[hoshsadiq]

FWIW when using docker import - image_name pointed to podman.socket it imports it (although the image itself doesn't get tagged)

[vrothberg]

I opened #11018 to fix the issue.

[vrothberg]

Got the same issue. Both for rootless and rootfull.

@hoshsadiq, can you share a reproducer? I am looking into adding a regression test but podam export; cat ... | podman import - does not trigger the error.

[hoshsadiq]

[ -r /tmp/DietPi_RPi-ARMv6-Buster.7z ] || curl -fsSL -o /tmp/DietPi_RPi-ARMv6-Buster.7z https://dietpi.com/downloads/images/DietPi_RPi-ARMv6-Buster.7z

tmp_dir="$(mktemp --directory -t dietpi.XXXXXXXX)"
trap '{ sudo umount -lf $tmp_dir/root || true; } && sudo rm -rf $tmp_dir' EXIT

/usr/bin/7za x /tmp/DietPi_RPi-ARMv6-Buster.7z -o"$tmp_dir" "*.img"

disk_img="$(basename "$(find "$tmp_dir" -name "*.img" -printf "%f")" .img)"
offset="$(fdisk -lu "${tmp_dir}/${disk_img}.img" | sed -n "s/\(^[^ ]*img2\)\s*\([0-9]*\)\s*\([0-9]*\)\s*\([0-9]*\).*/\2/p")"

podman run --rm -it -v "$tmp_dir:/workdir" -w /workdir --entrypoint sh multiarch/qemu-user-static -c 'cp /usr/bin/qemu-arm-static /workdir'

disk_root="$tmp_dir/root"
mkdir "$disk_root"
sudo mount -o loop,offset=$((offset * 512)) "${tmp_dir}/${disk_img}.img" "$disk_root"
sudo mv "$disk_root/etc/ld.so.preload" "$disk_root/etc/ld.so.preload.bak"
sudo cp "$tmp_dir/qemu-arm-static" "$disk_root/usr/bin"
sudo tar -C "$disk_root" -c . | podman import - "dietpi:$disk_img"

I've ripped this out of the full script I've got so a variable or two may be missing but I don't think it should.

[vrothberg]

Thanks everybody!