Podman unable to resolve fully-qualified image names from docker.io when using docker-compose

[sarayourfriend]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Steps to reproduce the issue:

1. Install rootless podman following the instructions at https://wiki.archlinux.org/title/Podman. Be sure to install podman-docker so that docker is aliased to podman.

2. Verify $DOCKER_HOST is correctly exported in shell environment (you can do this by doing something silly like setting it to an invalid value and ensuring that podman fails when running rootless podman commands).

3. Do not add docker.io to the list of registries for podman to resolve to

4. Use this simple docker-compose.yml with a fully-qualified docker.io image name and confirm that it fails with the error below:

version: '3'

services:
  database:
    image: docker.io/postgres:latest
    ports:
      - 5432:5432

    environment:
      POSTGRES_USER: username
      POSTGRES_PASSWORD: password
      POSTGRES_DB: default_database

Note: feel free to change the docker.io/postgres:latest to anything else you think might work like index.docker.io/... or docker.io/library/.... Regardless of these, none of them work.

Describe the results you received:

➜  ~ docker-compose up
[+] Running 0/0
 ⠿ database Error                                                                                            0.0s
Error response from daemon: failed to resolve image name: short-name "postgres:latest" did not resolve to an alias and no unqualified-search registries are defined in "/etc/containers/registries.conf"

Describe the results you expected:

Podman should successfully resolve the fully-qualified image name from docker.io in docker-compose.

Additional information you deem important (e.g. issue happens only occasionally):

This is something to do with podman's behavior under docker-compose specifically because running podman pull docker.io/postgres:latest works perfectly.

Note also that this only happens if you don't add docker.io to the list of registries. I know that adding docker.io to that list will resolve this for me locally but I'm specifically trying to use fully-qualified image names without needing to modify the podman registry configuration files.

Output of podman version:

➜  ~ podman version
Version:      3.4.4
API Version:  3.4.4
Go Version:   go1.17.4
Git Commit:   f6526ada1025c2e3f88745ba83b8b461ca659933
Built:        Thu Dec  9 13:30:40 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: /usr/bin/conmon is owned by conmon 1:2.1.0-1
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: bdb4f6e56cd193d40b75ffc9725d4b74a18cb33c'
  cpus: 8
  distribution:
    distribution: manjaro
    version: unknown
  eventLogger: journald
  hostname: sara-laptop
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.15.21-1-MANJARO
  linkmode: dynamic
  logDriver: journald
  memFree: 26377412608
  memTotal: 33439899648
  ociRuntime:
    name: crun
    package: /usr/bin/crun is owned by crun 1.4.2-1
    path: /usr/bin/crun
    version: |-
      crun version 1.4.2
      commit: f6fbc8f840df1a414f31a60953ae514fa497c748
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: /usr/bin/slirp4netns is owned by slirp4netns 1.1.12-1
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 1h 43m 51.67s (Approximately 0.04 days)
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries: {}
store:
  configFile: /home/sara/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/sara/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 0
  runRoot: /run/user/1000/containers
  volumePath: /home/sara/.local/share/containers/storage/volumes
version:
  APIVersion: 3.4.4
  Built: 1639074640
  BuiltTime: Thu Dec  9 13:30:40 2021
  GitCommit: f6526ada1025c2e3f88745ba83b8b461ca659933
  GoVersion: go1.17.4
  OsArch: linux/amd64
  Version: 3.4.4

Package info (e.g. output of rpm -q podman or apt list podman):

➜  ~ pacman -Q podman
podman 3.4.4-1

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

physical

[sarayourfriend]

Noting that I'm thinking this issue might be invalid with 4.0 around the corner which will automatically resolve images from docker.io (as far as I understand based on this comment: #11717 (comment))

I realized later that podman-compose exists and replacing docker-compose with podman-compose indeed resolves the fully-qualified docker.io image name. Not sure if that invalidates this issue or not. Feel free to close if so!

[baude]

this is not about dns name resolution. this is about "shortnames"; where Podman is trying to parse the image name input and figure out where it is coming from. This can be attributed to how your distribution choose to ship Podman. What happens if you try docker.io/library/postgres:latest ?

[sarayourfriend]

this is not about dns name resolution

I understand that 🙂

This can be attributed to how your distribution choose to ship Podman

I could be wrong about this, but I believe Arch's repositories are pulling directly from upstream. Or at least the configuration files are, according to the first sentence in this section on the wiki: https://wiki.archlinux.org/title/Podman#No_image_found

What happens if you try docker.io/library/postgres:latest

As I said in the issue description:

Note: feel free to change the docker.io/postgres:latest to anything else you think might work like index.docker.io/... or docker.io/library/.... Regardless of these, none of them work.

[mheon]

This should be resolved in v4.0 as v4.0 up hardcodes the docker.io registry for all Docker-compat API requests; still, while I think we've resolved the bug in a roundabout way, there's clearly still something wrong with the compat API's pull endpoint if it thinks that a fully-qualified name is a shortname.

[sarayourfriend]

Yeah, I am curious if there's a fix for non-compat version of Podman, specifically one's that strictly enforce OCI expectations outside of a docker-compat world (not sure how different they are at this point though).

If there's anything I can do to help debug this issue I'm happy to assist, just let me know.

[Luap99]

I think this is wrong on the docker-compose side, for example see #12320
The docker client removes docker.io domain before sending the name to the server. As @mheon said this is fixed in 4.0 so I close this issue.

[sarayourfriend]

Great! That confirms why it worked fine with podman-compose as well. Thanks y'all!