Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot pull from custom registry: x509: certificate signed by unknown authority #13823

Closed
shoeffner opened this issue Apr 11, 2022 · 17 comments
Closed
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. machine macos MacOS (OSX) related remote Problem is in podman-remote

Comments

@shoeffner
Copy link

/kind bug

Description

I can podman login into our internal harbor registry (say, registry.example.local), but I cannot pull images.
The registry has a certificate signed by our custom CA.

$ podman login
Username: 
Password:
$ podman pull registry.example.local/team/image:label
Trying to pull registry.example.local/team/image:label...
Error: initializing source docker://registry.example.local/team/image:label: pinging container registry registry.example.local: Get "https://registry.example.local/v2/": x509: certificate signed by unknown authority

Steps to reproduce the issue:

  1. Open browser at registry.example.local and download the CA pem certificate (I first used the one I have in my KeyChain)
  2. Install certificate as ~/.config/containers/certs.d/example-local.crt
  3. podman machine init
  4. podman machine start
  5. podman login
  6. podman pull registry.exmple.local/team/image:label

Alternatively, first ssh into the machine and then issue podman pull.

Describe the results you received:

$ podman pull registry.example.local/team/image:label
Trying to pull registry.example.local/team/image:label...
Error: initializing source docker://registry.example.local/team/image:label: pinging container registry registry.example.local: Get "https://registry.example.local/v2/": x509: certificate signed by unknown authority

Describe the results you expected:

Something like

% podman pull --tls-verify=false registry.example.local/team/image:label
Trying to pull registry.example.local/team/image:label...
Getting image source signatures
Copying blob sha256:9d2c24245c2d89f59843f45aa674a4d0f30e7a7...

but without --tls-verify=false.

Additional information you deem important (e.g. issue happens only occasionally):

I'm running macOS 12.3.1 on an M1.

The certificates should be correctly installed on machine init as per #12709, I verified this with

$ podman machine ssh
> ls /etc/containers/certs.d/
example-local.crt

Output of podman version:

Client:       Podman Engine
Version:      4.0.3
API Version:  4.0.3
Go Version:   go1.18
Built:        Fri Apr  1 17:28:59 2022
OS/Arch:      darwin/arm64

Server:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.16.14
Built:        Thu Mar  3 15:58:50 2022
OS/Arch:      linux/arm64

Output of podman info --debug:

host:
  arch: arm64
  buildahVersion: 1.24.1
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.0-2.fc35.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: '
  cpus: 1
  distribution:
    distribution: fedora
    variant: coreos
    version: "35"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 502
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
  kernel: 5.15.18-200.fc35.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 1197207552
  memTotal: 2048176128
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.4.2-1.fc35.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.4.2
      commit: f6fbc8f840df1a414f31a60953ae514fa497c748
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/502/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.aarch64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 9m 37.18s
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/user/502/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 4.0.2
  Built: 1646319530
  BuiltTime: Thu Mar  3 15:58:50 2022
  GitCommit: ""
  GoVersion: go1.16.14
  OsArch: linux/arm64
  Version: 4.0.2

Package info (e.g. output of rpm -q podman or apt list podman):

$ brew info podman

podman: stable 4.0.3 (bottled), HEAD
Tool for managing OCI containers and pods
https://podman.io/
/opt/homebrew/Cellar/podman/4.0.3 (172 files, 46.2MB) *
  Poured from bottle on 2022-04-11 at 10:04:08
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/podman.rb
License: Apache-2.0
==> Dependencies
Build: go ✔, go-md2man ✘
Required: qemu ✔
==> Options
--HEAD
        Install HEAD version
==> Caveats
zsh completions have been installed to:
  /opt/homebrew/share/zsh/site-functions

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Per README (as suggested in this template), the latest version is 4.0.0, I tested 4.0.3.
The items in the troubleshooting guide do not apply.

Additional environment details (AWS, VirtualBox, physical, etc.):

macOS Monterey 12.3.1, M1

@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Apr 11, 2022
@github-actions github-actions bot added macos MacOS (OSX) related remote Problem is in podman-remote labels Apr 11, 2022
@Luap99 Luap99 added the machine label Apr 11, 2022
@flouthoc
Copy link
Collaborator

Hi @shoeffner , Is this a self signed certificate ? Could you please elaborate about what do you mean by custom CA ? @mtrmac @vrothberg please correct me if i got this wrong afaik self-signed certs are still not supported, Could you try a solution using --tls-verify=false as suggested here: containers/buildah#3842 (comment)

@mtrmac
Copy link
Collaborator

mtrmac commented Apr 11, 2022

If this is local: Please compare the output of podman --log-level=debug for the podman login and podman pull commands; in particular there should be a Looking for TLS certificates and private keys in … message. Is the directory the same, and the expected path, in both commands?

But this is remote, isn’t it? If so, are the certificates present on the remote Linux computer? Compare #11507 .

@bo0ts
Copy link

bo0ts commented Apr 13, 2022

I'm running into the same problem with podman version 4.0.2 on macOS 12.3. A docker installation works fine. Here are the logs you requested:

~> podman login --log-level=debug my.registry.intern -u XXX -p XXX
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called login.PersistentPreRunE(podman login --log-level=debug my.registry.intern -u XXX -p XXX) 
DEBU[0000] SSH Ident Key "/Users/XXX/.ssh/podman-machine-default" SHA256:XXX ssh-ed25519 
DEBU[0000] Found SSH_AUTH_SOCK "/private/tmp/com.apple.launchd.6eGjCfIXV8/Listeners", ssh-agent signer(s) enabled 
DEBU[0000] DoRequest Method: GET URI: http://d/v4.0.2/libpod/_ping 
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf" 
DEBU[0000] Found credentials for my.registry.intern in credential helper containers-auth.json in file /Users/XXX/.config/containers/auth.json 
DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/my.registry.intern 
DEBU[0000] GET https://my.registry.intern/v2/ 
DEBU[0000] Ping https://my.registry.intern/v2/ status 401 
DEBU[0000] GET https://gitlab.registry.intern/jwt/auth?account=XXX&service=container_registry 
DEBU[0000] Increasing token expiration to: 60 seconds   
DEBU[0000] GET https://my.registry.intern/v2/ 
DEBU[0000] Stored credentials for my.registry.intern in credential helper containers-auth.json 
Login Succeeded!
DEBU[0000] Called login.PersistentPostRunE(podman login --log-level=debug my.registry.intern -u XXX -p XXX) 


~> podman pull --log-level=debug my.registry.intern/registry/images/my-image:latest
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called pull.PersistentPreRunE(podman pull --log-level=debug my.registry.intern/registry/images/my-image:latest) 
DEBU[0000] SSH Ident Key "/Users/XXX/.ssh/podman-machine-default" SHA256:XXX ssh-ed25519 
DEBU[0000] Found SSH_AUTH_SOCK "/private/tmp/com.apple.launchd.6eGjCfIXV8/Listeners", ssh-agent signer(s) enabled 
DEBU[0000] DoRequest Method: GET URI: http://d/v4.0.2/libpod/_ping 
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf" 
DEBU[0000] Found credentials for my.registry.intern in credential helper containers-auth.json in file /Users/XXX/.config/containers/auth.json 
DEBU[0000] No credentials matching docker.io found in /Users/XXX/.config/containers/auth.json 
DEBU[0000] No credentials matching docker.io found in /Users/XXX/.config/containers/auth.json 
DEBU[0000] No credentials matching docker.io found in /Users/XXX/.dockercfg 
DEBU[0000] No credentials for docker.io found           
DEBU[0000] Found credentials for some-other.registry.intern in credential helper containers-auth.json in file /Users/XXX/.docker/config.json 
DEBU[0000] DoRequest Method: POST URI: http://d/v4.0.2/libpod/images/pull 
Trying to pull my.registry.intern/registry/images/my-image:latest...
Error: initializing source docker://my.registry.intern/registry/images/my-image:latest: pinging container registry my.registry.intern: Get "https://my.registry.intern/v2/": x509: certificate signed by unknown authority

login works with HTTPS, but pull doesn't. The certificate is installed in the keychain, works with docker, and all other tools.

@shoeffner
Copy link
Author

@flouthoc the CA certificate is, as far as I know, self-signed; however, the server certificate is not, it is signed with the self-signed CA certificate.

I already tried --tls-verify=false, which works.

@mtrmac I am running a VM and, as listed above, the certificates are present on it:

$ podman machine ssh
> ls /etc/containers/certs.d/
example-local.crt

Here is the output as you requested:

$ podman login --log-level=debug registry.example.local
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called login.PersistentPreRunE(podman login --log-level=debug registry.example.local) 
DEBU[0000] SSH Ident Key "/Users/shoeffner/.ssh/podman-machine-default" SHA256:fcXJgc4<snip>Hkd0P2WDdw ssh-ed25519 
DEBU[0000] Found SSH_AUTH_SOCK "/private/tmp/com.apple.launchd.OynLOxn0RP/Listeners", ssh-agent signer(s) enabled 
DEBU[0000] DoRequest Method: GET URI: http://d/v4.0.3/libpod/_ping 
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf" 
DEBU[0000] No credentials matching registry.fme.lan found in /Users/shoeffner/.config/containers/auth.json 
DEBU[0000] No credentials matching registry.fme.lan found in /Users/shoeffner/.config/containers/auth.json 
DEBU[0000] No credentials matching registry.fme.lan found in /Users/shoeffner/.docker/config.json 
DEBU[0000] No credentials matching registry.fme.lan found in /Users/shoeffner/.dockercfg 
DEBU[0000] No credentials for registry.example.local found    
Username: shoeffner
Password: 
DEBU[0015] Looking for TLS certificates and private keys in /etc/docker/certs.d/registry.example.local
DEBU[0015] GET https://registry.example.local/v2/             
DEBU[0016] Ping https://registry.example.local/v2/ status 401 
DEBU[0016] GET https://registry.example.local/service/token?account=shoeffner&service=harbor-registry 
DEBU[0016] GET https://registry.example.local/v2/             
DEBU[0016] Stored credentials for registry.example.local in credential helper containers-auth.json 
Login Succeeded!
DEBU[0016] Called login.PersistentPostRunE(podman login --log-level=debug registry.example.local) 

and

$ podman pull --log-level=debug registry.example.local/team/image:label
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called pull.PersistentPreRunE(podman pull --log-level=debug registry.example.local/team/image:label) 
DEBU[0000] SSH Ident Key "/Users/shoeffner/.ssh/podman-machine-default" SHA256:fcXJgc4<snip>Hd26cPaQ7Hkd0P2WDdw ssh-ed25519 
DEBU[0000] Found SSH_AUTH_SOCK "/private/tmp/com.apple.launchd.OynLOxn0RP/Listeners", ssh-agent signer(s) enabled 
DEBU[0000] DoRequest Method: GET URI: http://d/v4.0.3/libpod/_ping 
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf" 
DEBU[0000] Found credentials for registry.example.local in credential helper containers-auth.json in file /Users/shoeffner/.config/containers/auth.json 
DEBU[0000] DoRequest Method: POST URI: http://d/v4.0.3/libpod/images/pull 
Trying to pull registry.example.local/team/image:label...
Error: initializing source docker://registry.example.local/team/image:label: pinging container registry registry.fme.lan: Get "https://registry.example.local/v2/": x509: certificate signed by unknown authority

With --tls-verify=false:

$ podman pull --log-level=debug --tls-verify=false registry.example.local/team/image:label
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called pull.PersistentPreRunE(podman pull --log-level=debug registry.example.local/team/image:label) 
DEBU[0000] SSH Ident Key "/Users/shoeffner/.ssh/podman-machine-default" SHA256:fcXJgc4<snip>Hd26cPaQ7Hkd0P2WDdw ssh-ed25519 
DEBU[0000] Found SSH_AUTH_SOCK "/private/tmp/com.apple.launchd.OynLOxn0RP/Listeners", ssh-agent signer(s) enabled 
DEBU[0000] DoRequest Method: GET URI: http://d/v4.0.3/libpod/_ping 
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf" 
DEBU[0000] Found credentials for registry.example.local in credential helper containers-auth.json in file /Users/shoeffner/.config/containers/auth.json 
DEBU[0000] DoRequest Method: POST URI: http://d/v4.0.3/libpod/images/pull 
Trying to pull registry.example.local/team/image:label...Getting image source signatures
Copying blob sha256:9d2c24245c2d89f59843f45aa674a4d0<snip>
Copying blob sha256:7b1a6ab2e44dbac178598dabe7cff59bd<snip>
<snip>
Copying config sha256:ecd89a2dd2e5cfa1e80d96129cfefb27<snip>
Writing manifest to image destination
Storing signatures
ecd89a2dd2e5cfa1e80d96129cfefb27<snip>
DEBU[0026] Called pull.PersistentPostRunE(podman pull --log-level=debug --tls-verify=false registry.example.local/team/image:label)

@mtrmac
Copy link
Collaborator

mtrmac commented Apr 19, 2022

$ podman login --log-level=debug registry.example.local
DEBU[0015] Looking for TLS certificates and private keys in /etc/docker/certs.d/registry.example.local

(works)
vs.

$ podman machine ssh
> ls /etc/containers/certs.d/
example-local.crt

So that’s not the right location for a certificate to be found (and indeed, the remote logs don’t contain a “Looking for TLS certificates” log entry. Compare also https://github.com/containers/image/blob/main/docs/containers-certs.d.5.md#directory-structure .

(I haven’t looked into how that incorrect destination was created, so I don’t know whether that might be user error or a bug in podman machine, or something else.)

@shoeffner
Copy link
Author

Alright, I see, it has to be in /etc/containers/certs.d/registry.example.local/example-local.crt, sorry for stealing your time by not reading properly... :-(

Now the login actually finds the certificate:

$ podman login registry.example.local
...
DEBU[0070] Looking for TLS certificates and private keys in /Users/shoeffner/.config/containers/certs.d/registry.example.local 
DEBU[0070]  crt: /Users/shoeffner/.config/containers/certs.d/registry.example.local/example-local.crt 
DEBU[0070] GET https://registry.example.local/v2/   
...

And the pull works. Thank you!

I wonder why the login worked and the pull didn't though... Shouldn't the login reject the non-verifiable cert?

@shoeffner
Copy link
Author

Also, I just realized that the podman pull command does not log the cert lookups, which the login does, but I guess it's fine (plus, the lookup directory changed?! I don't know how I managed to do that...). But it works, thank you again!

@rhatdan
Copy link
Member

rhatdan commented Apr 19, 2022

Since this looks like it is working properly, I am going to close.

@rhatdan rhatdan closed this as completed Apr 19, 2022
@eidottermihi
Copy link

I'm still experiencing this problem :(

I stored the CA certificate according to the docs (https://github.com/containers/image/blob/main/docs/containers-certs.d.5.md#directory-structure), and podman login correctly pickups the custom certificate:

PS C:\Users\michael.prankl> podman login registry.mycompany.com --log-level=debug
time="2022-12-02T08:58:19+01:00" level=info msg="C:\\Program Files\\RedHat\\Podman\\podman.exe filtering at log level debug"
time="2022-12-02T08:58:19+01:00" level=debug msg="Called login.PersistentPreRunE(C:\\Program Files\\RedHat\\Podman\\podman.exe login registry.mycompany.com --log-level=debug)"
time="2022-12-02T08:58:19+01:00" level=debug msg="SSH Ident Key \"C:\\\\Users\\\\michael.prankl\\\\.ssh\\\\podman-machine-default\" SHA256:hnwVwQwuZlStVqh3ikEF/LcyrsUdFbt+BwAzeBuqDuQ ssh-ed25519"
time="2022-12-02T08:58:19+01:00" level=debug msg="DoRequest Method: GET URI: http://d/v4.3.1/libpod/_ping"
time="2022-12-02T08:58:19+01:00" level=debug msg="Loading registries configuration \"C:\\\\Users\\\\michael.prankl\\\\.config\\\\containers\\\\registries.conf\""
time="2022-12-02T08:58:19+01:00" level=debug msg="No credentials matching registry.mycompany.com found in C:\\Users\\michael.prankl\\.config\\containers\\auth.json"
time="2022-12-02T08:58:19+01:00" level=debug msg="No credentials matching registry.mycompany.com found in C:\\Users\\michael.prankl\\.config\\containers\\auth.json"
time="2022-12-02T08:58:19+01:00" level=debug msg="No credentials matching registry.mycompany.com found in C:\\Users\\michael.prankl\\.docker\\config.json"
time="2022-12-02T08:58:19+01:00" level=debug msg="No credentials matching registry.mycompany.com found in C:\\Users\\michael.prankl\\.dockercfg"
time="2022-12-02T08:58:19+01:00" level=debug msg="No credentials for registry.mycompany.com found"
Username: michael.prankl
Password:
time="2022-12-02T08:58:26+01:00" level=debug msg="Looking for TLS certificates and private keys in C:\\Users\\michael.prankl\\.config\\containers\\certs.d\\registry.mycompany.com"
time="2022-12-02T08:58:26+01:00" level=debug msg=" crt: C:\\Users\\michael.prankl\\.config\\containers\\certs.d\\registry.mycompany.com\\mycompany.crt.crt"
time="2022-12-02T08:58:26+01:00" level=debug msg="GET https://registry.mycompany.com/v2/"
time="2022-12-02T08:58:26+01:00" level=debug msg="Ping https://registry.mycompany.com/v2/ status 200"
time="2022-12-02T08:58:26+01:00" level=debug msg="GET https://registry.mycompany.com/v2/"
time="2022-12-02T08:58:27+01:00" level=debug msg="Stored credentials for registry.mycompany.com in credential helper containers-auth.json"
Login Succeeded!
time="2022-12-02T08:58:27+01:00" level=debug msg="Called login.PersistentPostRunE(C:\\Program Files\\RedHat\\Podman\\podman.exe login registry.mycompany.com --log-level=debug)"

However, podman pull still fails with x509: certificate signed by unknown authority:

PS C:\Users\michael.prankl> podman pull registry.mycompany.com/metabase/metabase:latest --log-level=trace
time="2022-12-02T08:58:46+01:00" level=info msg="C:\\Program Files\\RedHat\\Podman\\podman.exe filtering at log level trace"
time="2022-12-02T08:58:46+01:00" level=debug msg="Called pull.PersistentPreRunE(C:\\Program Files\\RedHat\\Podman\\podman.exe pull registry.mycompany.com/metabase/metabase:latest --log-level=trace)"
time="2022-12-02T08:58:46+01:00" level=debug msg="SSH Ident Key \"C:\\\\Users\\\\michael.prankl\\\\.ssh\\\\podman-machine-default\" SHA256:hnwVwQwuZlStVqh3ikEF/LcyrsUdFbt+BwAzeBuqDuQ ssh-ed25519"
time="2022-12-02T08:58:46+01:00" level=debug msg="DoRequest Method: GET URI: http://d/v4.3.1/libpod/_ping"
time="2022-12-02T08:58:46+01:00" level=debug msg="Loading registries configuration \"C:\\\\Users\\\\michael.prankl\\\\.config\\\\containers\\\\registries.conf\""
time="2022-12-02T08:58:46+01:00" level=debug msg="Found credentials for registry.mycompany.com in credential helper containers-auth.json in file C:\\Users\\michael.prankl\\.config\\containers\\auth.json"
time="2022-12-02T08:58:46+01:00" level=debug msg="No credentials matching capregistry.muenchen.de found in C:\\Users\\michael.prankl\\.config\\containers\\auth.json"
time="2022-12-02T08:58:46+01:00" level=debug msg="No credentials matching capregistry.muenchen.de found in C:\\Users\\michael.prankl\\.config\\containers\\auth.json"
time="2022-12-02T08:58:46+01:00" level=debug msg="Found an empty credential entry \"capregistry.muenchen.de\" in \"C:\\\\Users\\\\michael.prankl\\\\.docker\\\\config.json\" (an unhandled credential helper marker?), moving on"
time="2022-12-02T08:58:46+01:00" level=debug msg="No credentials matching capregistry.muenchen.de found in C:\\Users\\michael.prankl\\.dockercfg"
time="2022-12-02T08:58:46+01:00" level=debug msg="No credentials for capregistry.muenchen.de found"
time="2022-12-02T08:58:46+01:00" level=debug msg="DoRequest Method: POST URI: http://d/v4.3.1/libpod/images/pull"
Trying to pull registry.mycompany.com/metabase/metabase:latest...
Error: initializing source docker://registry.mycompany.com/metabase/metabase:latest: pinging container registry registry.mycompany.com: Get "https://registry.mycompany.com/v2/": x509: certificate signed by unknown authority

podman version:

Client:       Podman Engine
Version:      4.3.1
API Version:  4.3.1
Go Version:   go1.18.5
Git Commit:   814b7b003cc630bf6ab188274706c383f9fb9915
Built:        Thu Nov 10 15:18:45 2022
OS/Arch:      windows/amd64

Server:       Podman Engine
Version:      4.3.1
API Version:  4.3.1
Go Version:   go1.18.7
Built:        Fri Nov 11 16:24:13 2022
OS/Arch:      linux/amd64

my registries.conf:

unqualified-search-registries = ['docker.io']

[[registry]]
location = "docker.io"

[[registry.mirror]]
location = "registry.mycompany.com"

@vrothberg
Copy link
Member

@eidottermihi did you configure the certs in the Linux VM?

@eidottermihi
Copy link

@vrothberg not sure what you mean by Linux VM, I am using Podman Desktop 0.10 + podman 4.3.1 on Windows.

I stored the cert in %HOME%\.config\containers\certs.d\ , as you can see in the logs from podman login:

time="2022-12-02T08:58:26+01:00" level=debug msg="Looking for TLS certificates and private keys in C:\\Users\\michael.prankl\\.config\\containers\\certs.d\\registry.mycompany.com"
time="2022-12-02T08:58:26+01:00" level=debug msg=" crt: C:\\Users\\michael.prankl\\.config\\containers\\certs.d\\registry.mycompany.com\\mycompany.crt.crt"

@vrothberg
Copy link
Member

Podman Desktop is creating a Linux VM where the containers (and Podman) are actually running. Hence, the certs in your Windows Home directory won't be read.

@benoitf, is there a best-practice in Podman Desktop to set the certs?

@eidottermihi
Copy link

I understand, but that is rather weird than:

podman login uses the certs stored in my windows home directory, podman pull does not?

@benoitf
Copy link
Contributor

benoitf commented Dec 2, 2022

no, for now, Podman Desktop reads the Operating System certificates for its own operations but is not propagating these to the podman machine. We were planning some documentation first.

@eidottermihi you would need to add these certificates into the 'podman machine' that you can reach using the podman machine ssh command.

I think it's because podman login is executed on your system (it will create auth.json on your Windows) while podman pull is executed within the VM

@benoitf
Copy link
Contributor

benoitf commented Dec 2, 2022

also if could be parameter of the podman machine (or in containers.conf to copy certificates into the machine when creating it)

@eidottermihi
Copy link

I followed the instructions in https://github.com/containers/podman/blob/main/docs/tutorials/podman-install-certificate-authority.md to install the CA certs inside the podman machine, now it is working as expected :)

@DarekDan
Copy link

I followed the instructions in https://github.com/containers/podman/blob/main/docs/tutorials/podman-install-certificate-authority.md to install the CA certs inside the podman machine, now it is working as expected :)

That is the key. The custom CA must be inside the machine in a directory by the same name as the regsitry, i.e docker.io

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 6, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. machine macos MacOS (OSX) related remote Problem is in podman-remote
Projects
None yet
Development

No branches or pull requests

10 participants