podman pull doesn't use default --cert-dir as indicated in the manual

[micah]

/kind bug

Description

The manual page for podman pull details this option:

   --cert-dir=path
       Use certificates at path (*.crt, *.cert, *.key) to connect to the registry.  Default certificates directory is /etc/containers/certs.d. (Not available for remote commands)

however, it doesn't appear that --cert-dir=/etc/containers/certs.d is actually the default for this command.

Steps to reproduce the issue:

1. Create a registry with a certificate signed by a private CA (for example by deploying https://github.com/rpardini/docker-registry-proxy)

2. put the CA certificate in /etc/containers/certs.d/additional.crt

3. attempt to pull from that registry

4. try again, but this time pass -certs-dir=/etc/containers/certs.d to the pull command

Describe the results you received:

I have a CA cert in /etc/containers/certs.d/additional.crt and podman pull, without the flag specified, complains about the validity of the x509 certificate:

Error: Error initializing source docker://registry.example.com/docker/grafana:master: error pinging docker registry registry.example.com Get "https://registry.example.com/v2/": x509: certificate signed by unknown authority

Describe the results you expected:

If I pass manually --cert-dir=/etc/containers/certs.d then the pull does manage to validate the certificate and pull properly:

podman pull --cert-dir=/etc/containers/certs.d registry.example.com/docker/grafana:master
Trying to pull registry.example.com/docker/grafana:master...
Getting image source signatures
Copying blob 74dd2fd0d41c skipped: already exists  
Copying blob a076a628af6f [--------------------------------------] 0.0b / 0.0b
Copying blob 22c2d8e413a3 [--------------------------------------] 0.0b / 0.0b
Copying config 19d65b7b68 done  
Writing manifest to image destination
Storing signatures
19d65b7b68ed4b0b3c223b861d6f27cb502215d367b8c072579266af532841d9
root@host3:/home/vagrant# 

I would expect that I would not need to pass --cert-dir as this is supposedly the default option.

Additional information you deem important (e.g. issue happens only occasionally):

I also see no place where I can configure this option as a default other than having to put it in every pull command.

Output of podman version:

root@host3:/home/vagrant# podman version
Version:      3.0.1
API Version:  3.0.0
Go Version:   go1.14
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64

Output of podman info --debug:

root@host3:/home/vagrant# podman info --debug
host:
  arch: amd64
  buildahVersion: 1.19.4
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: 'conmon: /usr/libexec/podman/conmon'
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.27, commit: '
  cpus: 1
  distribution:
    distribution: debian
    version: "10"
  eventLogger: journald
  hostname: host3
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 4.19.0-16-amd64
  linkmode: dynamic
  memFree: 439635968
  memTotal: 3081691136
  ociRuntime:
    name: crun
    package: 'crun: /usr/bin/crun'
    path: /usr/bin/crun
    version: |-
      crun version 0.18.1-7931a-dirty
      commit: 7931a1eab0590eff4041c1f74e2844b297c31cea
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: true
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    selinuxEnabled: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 26m 56.26s
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 4
    paused: 0
    running: 4
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 6
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.0.0
  Built: 0
  BuiltTime: Thu Jan  1 00:00:00 1970
  GitCommit: ""
  GoVersion: go1.14
  OsArch: linux/amd64
  Version: 3.0.1

Package info (e.g. output of rpm -q podman or apt list podman):

root@host3:/home/vagrant# apt list podman
Listing... Done
podman/unknown,now 100:3.0.1-2 amd64 [installed]
podman/unknown 100:3.0.1-2 arm64
podman/unknown 100:3.0.1-2 armhf
podman/unknown 100:3.0.1-2 ppc64el
podman/unknown 100:3.0.1-2 s390x
root@host3:/home/vagrant# 

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes

[vrothberg]

That should be fixed with #10147.

[vrothberg]

Thanks for the issue, @micah !

[vrothberg]

Apologies for not going back to the issue before, @micah.

While we need to update the podman-pull man page, the issue is that the certs.d directory does not comply with the expected structure (see man containers-certs.d). It's /etc/containers/certs.d/additional.crt but should be /etc/containers/certs.d/registry.example.com/additional.crt (note that /registry.example.com/).

I am going to close the issue and prepare a follow-up PR to massage the man pages.

[jonyhy96]

@vrothberg this issue seems not solved
https://github.com/containers/podman/blob/main/cmd/podman/images/pull.go#L112 it still not use /etc/containers/certs.d as default value for --cert-dir. should be like

flags.StringVar(&pullOptions.CertDir, certDirFlagName, "/etc/containers/certs.d", "`Pathname` of a directory containing TLS certificates and keys")

[vrothberg]

@jonyhy96, https://github.com/containers/image/blob/main/docker/docker_client.go#L65-L68 is the place to look at.