-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error downloading kic artifacts #14664
Comments
Loading images from cache is just a bonus, it is supposed to be able to pull them from the registry otherwise.
Can you do something simple like |
Actually I have no idea if it works with rootless podman, previously it was recommended to use the regular:
https://minikube.sigs.k8s.io/docs/drivers/podman/ Apparently rootless podman only works with containerd |
There is like three experimental things at one time here:
|
➜ ~ podman pull k8s.gcr.io/pause:3.7
Trying to pull k8s.gcr.io/pause:3.7...
Getting image source signatures
Copying blob sha256:aff472d3f83edbbc738d035ea53108fcb1e10564aaf0c8d3d6576a02cc2a5679
Copying blob sha256:aff472d3f83edbbc738d035ea53108fcb1e10564aaf0c8d3d6576a02cc2a5679
Copying config sha256:e5a475a0380575fb5df454b2e32bdec93e1ec0094d8a61e895b41567cb884550
Writing manifest to image destination
Storing signatures
e5a475a0380575fb5df454b2e32bdec93e1ec0094d8a61e895b41567cb884550
I'm trying to narrow down the issue I deleted an create a new podman machine, rootful this time rootful podman
rootful minikube start --driver=podman
😄 minikube v1.26.0 on Darwin 12.5 (arm64)
✨ Using the podman (experimental) driver based on existing profile
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
E0728 16:13:54.027788 20879 cache.go:203] Error downloading kic artifacts: not yet implemented, see issue #8426
🔄 Restarting existing podman container for "minikube" ...
🤦 StartHost failed, but will try again: podman inspect ip minikube: podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:
stderr:
Error: inspecting object: no such container "minikube"
🔄 Restarting existing podman container for "minikube" ...
😿 Failed to start podman container. Running "minikube delete" may fix it: podman inspect ip minikube: podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:
stderr:
Error: inspecting object: no such container "minikube"
❌ Exiting due to GUEST_PROVISION: Failed to start host: podman inspect ip minikube: podman container inspect -f minikube: exit status 125
stdout:
stderr:
Error: inspecting object: no such container "minikube"
╭───────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 😿 If the above advice does not help, please let us know: │
│ 👉 https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────╯ rootful with cri-o
rootful with containerd➜ ~ minikube start --driver=podman --container-runtime=containerd
😄 minikube v1.26.0 on Darwin 12.5 (arm64)
✨ Using the podman (experimental) driver based on existing profile
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
💾 Downloading Kubernetes v1.24.1 preload ...
> preloaded-images-k8s-v18-v1...: 411.49 MiB / 411.49 MiB 100.00% 2.87 MiB
E0728 16:17:24.815749 20940 cache.go:203] Error downloading kic artifacts: not yet implemented, see issue #8426
🔄 Restarting existing podman container for "minikube" ...
🤦 StartHost failed, but will try again: podman inspect ip minikube: podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:
stderr:
Error: inspecting object: no such container "minikube"
🔄 Restarting existing podman container for "minikube" ...
😿 Failed to start podman container. Running "minikube delete" may fix it: podman inspect ip minikube: podman container inspect -f {{.NetworkSettings.IPAddress}} minikube: exit status 125
stdout:
stderr:
Error: inspecting object: no such container "minikube"
❌ Exiting due to GUEST_PROVISION: Failed to start host: podman inspect ip minikube: podman container inspect -f minikube: exit status 125
stdout:
stderr:
Error: inspecting object: no such container "minikube"
╭───────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 😿 If the above advice does not help, please let us know: │
│ 👉 https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────╯ cannot generate logs➜ ~ minikube logs --file=logs.txt
❌ Exiting due to GUEST_STATUS: state: unknown state "minikube": podman container inspect minikube --format=: exit status 125
stdout:
stderr:
Error: inspecting object: no such container "minikube"
╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ 😿 If the above advice does not help, please let us know: │
│ 👉 https://github.com/kubernetes/minikube/issues/new/choose │
│ │
│ Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue. │
│ Please also attach the following file to the GitHub issue: │
│ - /var/folders/9v/4dpzzrw56m1glmbj5zl6xlvm0000gn/T/minikube_logs_8f6474a291f68fa61b92987d0579232b5754d600_0.log │
│ │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ it seems to expect the minikube container to be there already, but isn't
I've |
Actually I meant the results from the This setup/combination is supposed to work: Podman Engine (linux), rootful podman, rootful cri-o Not sure what the status with the preload/cache was, I think it was left as half-implemented or something ? |
I have the same running podman in archlinux. E0916 20:41:26.765749 4830 cache.go:203] Error downloading kic artifacts: not yet implemented, see issue #8426 But is seems to run just fine beside the error: ~ » minikube profile list the same if I start with containerd or docker |---------|-----------|------------|--------------|------|---------|---------|-------|
|
I think the error just means that it failed to load the cached image, and will start to pull it from the network again. The runtime doesn't matter (crio), only the driver engine (podman). It's very similar, but different levels/caches...
Actually the cache is also the same for all container runtimes, the runtime only matters for the "preload":
It can be forced to only use the cache, with the
|
Typical cache contents: (with preload true+false)
$ docker images gcr.io/k8s-minikube/kicbase
REPOSITORY TAG IMAGE ID CREATED SIZE
gcr.io/k8s-minikube/kicbase v0.0.34 5f58fddaff43 2 weeks ago 1.14GB
$ minikube ssh docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.k8s.io/kube-apiserver v1.25.0 4d2edfd10d3e 3 weeks ago 128MB
registry.k8s.io/kube-scheduler v1.25.0 bef2cf311509 3 weeks ago 50.6MB
registry.k8s.io/kube-controller-manager v1.25.0 1a54c86c03a6 3 weeks ago 117MB
registry.k8s.io/kube-proxy v1.25.0 58a9a0c6d96f 3 weeks ago 61.7MB
registry.k8s.io/pause 3.8 4873874c08ef 3 months ago 711kB
registry.k8s.io/etcd 3.5.4-0 a8a176a5d5d6 3 months ago 300MB
registry.k8s.io/coredns/coredns v1.9.3 5185b96f0bec 3 months ago 48.8MB
k8s.gcr.io/pause 3.6 6270bb605e12 12 months ago 683kB
gcr.io/k8s-minikube/storage-provisioner v5 6e38f40d628d 17 months ago 31.5MB The k8s.gcr.io/pause:3.6 is due to a bug. |
The workaround would be to use $ minikube start --driver=podman --download-only
$ minikube start --help | grep kicbase
--base-image='gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c':
$ sudo podman load <~/.minikube/cache/kic/amd64/kicbase_v0.0.34@sha256_f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c.tar
Loaded image(s): gcr.io/k8s-minikube/kicbase:v0.0.34
$ minikube start --driver=podman Or fix the minikube code to be less docker-centric, which is unfortunately a feature inherited from go-containerregistry. (if it matters, the cache images are stored using the equivalent of |
sorry for the silence @afbjorklund I manage to run minikube w/ podman both rootless and rootful setup brew install minikube
brew install podman
podman machine init --cpus 2 ➜ ~ minikube version
minikube version: v1.27.0
commit: 4243041b7a72319b9be7842a7d34b6767bbdac2b ➜ ~ podman version
Client: Podman Engine
Version: 4.2.1
API Version: 4.2.1
Go Version: go1.18.6
Built: Tue Sep 6 22:16:02 2022
OS/Arch: darwin/arm64
Server: Podman Engine
Version: 4.2.0
API Version: 4.2.0
Go Version: go1.18.4
Built: Thu Aug 11 17:43:11 2022
OS/Arch: linux/arm64 'c. [email protected]
,xNMM. ---------------------------------------
.OMMMMo OS: macOS 12.6 21G115 arm64
OMMM0, Host: MacBookAir10,1
.;loddo:' loolloddol;. Kernel: 21.6.0
cKMMMMMMMMMMNWMMMMMMMMMM0: Uptime: 3 hours, 12 mins
.KMMMMMMMMMMMMMMMMMMMMMMMWd. Packages: 54 (brew)
XMMMMMMMMMMMMMMMMMMMMMMMX. Shell: zsh 5.8.1
;MMMMMMMMMMMMMMMMMMMMMMMM: Resolution: 1440x900
:MMMMMMMMMMMMMMMMMMMMMMMM: DE: Aqua
.MMMMMMMMMMMMMMMMMMMMMMMMX. WM: Quartz Compositor
kMMMMMMMMMMMMMMMMMMMMMMMMWd. WM Theme: Blue (Dark)
.XMMMMMMMMMMMMMMMMMMMMMMMMMMk Terminal: iTerm2
.XMMMMMMMMMMMMMMMMMMMMMMMMK. Terminal Font: Monaco 12
kMMMMMMMMMMMMMMMMMMMMMMd CPU: Apple M1
;KMMMMMMMWXXWMMMMMMMk. GPU: Apple M1
.cooc,. .,coo:. Memory: 1082MiB / 8192MiB
rootful podman machine set --rootful
podman system connection default podman-machine-default-root
podman machine start
minikube start --driver=podman --container-runtime=cri-o ➜ ~ minikube start --driver=podman --container-runtime=cri-o
😄 minikube v1.27.0 on Darwin 12.6 (arm64)
❗ Kubernetes 1.25.0 has a known issue with resolv.conf. minikube is using a workaround that should work for most use cases.
❗ For more information, see: https://github.com/kubernetes/kubernetes/issues/112135
✨ Using the podman (experimental) driver based on user configuration
📌 Using Podman driver with root privileges
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
💾 Downloading Kubernetes v1.25.0 preload ...
> preloaded-images-k8s-v18-v1...: 340.75 MiB / 340.75 MiB 100.00% 1.31 Mi
> gcr.io/k8s-minikube/kicbase: 348.00 MiB / 348.00 MiB 100.00% 1.01 MiB p
E1003 15:51:21.375384 27482 cache.go:203] Error downloading kic artifacts: not yet implemented, see issue #8426
🔥 Creating podman container (CPUs=2, Memory=1956MB) ...
🎁 Preparing Kubernetes v1.25.0 on CRI-O 1.24.2 ...
E1003 15:54:12.003380 27482 start.go:129] Unable to get host IP: RoutableHostIPFromInside is currently only implemented for linux
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔗 Configuring CNI (Container Networking Interface) ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default rootless podman system connection default podman-machine-default
minikube config set rootless true
podman machine start
minikube start --driver=podman --container-runtime=containerd ➜ ~ minikube start --driver=podman --container-runtime=containerd
😄 minikube v1.27.0 on Darwin 12.6 (arm64)
▪ MINIKUBE_ROOTLESS=true
❗ Kubernetes 1.25.0 has a known issue with resolv.conf. minikube is using a workaround that should work for most use cases.
❗ For more information, see: https://github.com/kubernetes/kubernetes/issues/112135
✨ Using the podman (experimental) driver based on user configuration
📌 Using rootless Podman driver
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
💾 Downloading Kubernetes v1.25.0 preload ...
> preloaded-images-k8s-v18-v1...: 340.35 MiB / 340.35 MiB 100.00% 2.32 Mi
E1003 16:33:58.875113 34042 cache.go:203] Error downloading kic artifacts: not yet implemented, see issue #8426
🔥 Creating podman container (CPUs=2, Memory=1956MB) ...
📦 Preparing Kubernetes v1.25.0 on containerd 1.6.8 ...
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔗 Configuring CNI (Container Networking Interface) ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟 Enabled addons: storage-provisioner, default-storageclass
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default in both cases I get I did not try your workaround :( |
@afbjorklund ➜ minikube_setup git:(master) ✗ minikube start --download-only
😄 minikube v1.28.0 on Darwin 13.0.1 (arm64)
▪ MINIKUBE_ROOTLESS=true
✨ Using the podman (experimental) driver based on existing profile
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
> kubectl.sha256: 64 B / 64 B [-------------------------] 100.00% ? p/s 0s
> kubectl: 47.38 MiB / 47.38 MiB [-------------] 100.00% 6.31 MiB p/s 7.7s
✅ Download complete!
➜ minikube_setup git:(master) ✗ minikube start --help | grep kicbase
--base-image='gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c':
--base-image='gcr.io/k8s-minikube/kicbase:v0.0.36@sha256:8debc1b6a335075c5f99bfbf131b4f5566f68c6500dc5991817832e55fcc9456':
zsh: no such file or directory: --base-image=gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c:
➜ minikube_setup git:(master) ✗ minikube start --help | grep kicbase
--base-image='gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c'
--base-image='gcr.io/k8s-minikube/kicbase:v0.0.36@sha256:8debc1b6a335075c5f99bfbf131b4f5566f68c6500dc5991817832e55fcc9456':
zsh: no such file or directory: --base-image=gcr.io/k8s-minikube/kicbase:v0.0.34@sha256:f2a1e577e43fd6769f35cdb938f6d21c3dacfd763062d119cade738fa244720c |
I am having same problems on Linux.. podman driver does not work rootless (tried without container-runtime, with cri-o and containerd). |
somehow after 1'000'000 attempts I managed, plus I added this in my .zshrc (macos) export DOCKER_HOST=unix:///run/podman/podman.sock
export DOCKER_HOST=unix:///var/run/nerdctl.sock ➜ ~ podman machine start
Starting machine "podman-machine-default"
Waiting for VM ...
Mounting volume... /Users/yakforward:/Users/yakforward
This machine is currently configured in rootless mode. If your containers
require root permissions (e.g. ports < 1024), or if you run into compatibility
issues with non-podman clients, you can switch using the following command:
podman machine set --rootful
API forwarding listening on: /var/run/docker.sock
Docker API clients default to this address. You do not need to set DOCKER_HOST.
Machine "podman-machine-default" started successfully
➜ ~ podman images
REPOSITORY TAG IMAGE ID CREATED SIZE
<none> <none> 4117dd375c44 2 days ago 547 MB
<none> <none> 3f6ea93abaea 2 days ago 547 MB
docker.io/library/dart stable 51825d64cf05 8 days ago 516 MB
gcr.io/k8s-minikube/kicbase v0.0.36 c87ac1e75807 5 weeks ago 1.03 GB
➜ ~ minikube start
😄 minikube v1.28.0 on Darwin 13.0.1 (arm64)
▪ MINIKUBE_ROOTLESS=true
✨ Using the podman (experimental) driver based on existing profile
👍 Starting control plane node minikube in cluster minikube
🚜 Pulling base image ...
E1204 22:40:46.007070 29249 cache.go:203] Error downloading kic artifacts: not yet implemented, see issue #8426
🔄 Restarting existing podman container for "minikube" ...
📦 Preparing Kubernetes v1.25.3 on containerd 1.6.9 ...
🔗 Configuring CNI (Container Networking Interface) ...
🔎 Verifying Kubernetes components...
▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
▪ Using image docker.io/kubernetesui/dashboard:v2.7.0
▪ Using image docker.io/kubernetesui/metrics-scraper:v1.0.8
💡 Some dashboard features require the metrics-server addon. To enable all features please run:
minikube addons enable metrics-server
🌟 Enabled addons: storage-provisioner, default-storageclass, dashboard
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
➜ ~ minikube dashboard
🤔 Verifying dashboard health ...
🚀 Launching proxy ...
🤔 Verifying proxy health ...
🎉 Opening http://127.0.0.1:56160/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ in your default browser... but without knowing exactly how to make it work, I won't call the issue fixed |
Hello, I had this issue on my Linux box... until I figured out it was caused by a disk quota issue. Indeed minicube downloads bunch of things in $HOME/.minibube and in $HOME/.kube "invisible" directories. To by-pass my quota issue (our home directories are hosted by a NetApp with company-wide enforced quotas), I just did that:
then re-ran: => it worked fine! Of course that assumes that there's some room on a local disk/volume to create. Baseline: minikube should corretcly warn about disk space / quotas exhaustion, I've seen couple of similar reports with unrelated solutions (network stuff etc), so it would be great if dev team could improve disk space-based reporting/output to help beginners |
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten |
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /close not-planned |
@k8s-triage-robot: Closing this issue, marking it as "Not Planned". In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
What Happened?
steps to reproduce
minikube start --driver=podman --container-runtime=cri-o
note
see issue #8426
closed at the time of writing, but still active
cc @medyagh
Attach the log file
I got an error running
minikube logs --file=log.txt
too,please see the logs below
logs
Operating System
macOS (Default)
Driver
Podman
The text was updated successfully, but these errors were encountered: