[Podman v4] minikube delete does not remove Podman's minikube network

[AkihiroSuda]

What Happened?

minikube delete does not remove Podman's minikube network.

Host: minikube 1.25.2, Podman 4.0.2, Ubuntu 21.10.

Attach the log file

$ minikube start --driver=podman
* minikube v1.25.2 on Ubuntu 21.10
* Using the podman driver based on user configuration
...
* Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default


$ minikube delete
* Deleting "minikube" in podman ...
* Deleting container "minikube" ...
* Removing /home/suda/.minikube/machines/minikube ...
* Removed all traces of the "minikube" cluster.

$ sudo podman network inspect minikube
[
     {
          "name": "minikube",
          "id": "5086431107cae0151fdd8a231177374dfd063cb427d03273af07d527a23c27d0",
          "driver": "bridge",
          "network_interface": "cni-podman1",
          "created": "2022-03-26T23:56:35.521691614+09:00",
          "subnets": [
               {
                    "subnet": "192.168.49.0/24",
                    "gateway": "192.168.49.1"
               }
          ],
          "ipv6_enabled": false,
          "internal": false,
          "dns_enabled": true,
          "labels": {
               "created_by.minikube.sigs.k8s.io": "true"
          },
          "ipam_options": {
               "driver": "host-local"
          }
     }
]

Operating System

Ubuntu

Driver

Podman

[spowelljr]

Hi @AkihiroSuda, thanks for reporting your issue with minikube!

This is definitely an issue, thanks for bringing it to our attention!

[afbjorklund]

I'm seeing deletion of all volumes and networks here (with 3.4.2), even if there is no user output.

minikube --alsologtostderr delete

🔥  Deleting container "minikube" ...
I0329 20:07:56.980382   27150 cli_runner.go:133] Run: sudo -n podman container inspect minikube --format={{.State.Status}}
I0329 20:07:57.149511   27150 cli_runner.go:133] Run: sudo -n podman exec --privileged -t minikube /bin/bash -c "sudo init 0"
I0329 20:07:58.348382   27150 cli_runner.go:133] Run: sudo -n podman container inspect minikube --format={{.State.Status}}
I0329 20:07:58.509709   27150 oci.go:670] temporary error: container minikube status is Running but expect it to be exited
I0329 20:07:58.509808   27150 oci.go:676] Successfully shutdown container minikube
I0329 20:07:58.509994   27150 cli_runner.go:133] Run: sudo -n podman rm -f -v minikube
I0329 20:08:07.904329   27150 cli_runner.go:186] Completed: sudo -n podman rm -f -v minikube: (9.394279074s)
I0329 20:08:07.904350   27150 volumes.go:79] trying to delete all podman volumes with label name.minikube.sigs.k8s.io=minikube
I0329 20:08:07.904404   27150 cli_runner.go:133] Run: sudo -n podman volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}
I0329 20:08:08.095331   27150 cli_runner.go:133] Run: sudo -n podman volume rm --force minikube
I0329 20:08:08.538827   27150 cli_runner.go:133] Run: sudo -n podman network ls --filter=label=created_by.minikube.sigs.k8s.io --format {{.Name}}
I0329 20:08:08.697387   27150 cli_runner.go:133] Run: sudo -n podman network inspect minikube --format "{{range .plugins}}{{if eq .type "bridge"}}{{(index (index .ipam.ranges 0) 0).subnet}},{{(index (index .ipam.ranges 0) 0).gateway}}{{end}}{{end}}"
I0329 20:08:08.873108   27150 cli_runner.go:133] Run: sudo -n podman network rm minikube

Maybe the filters got broken again ?

[spowelljr]

Maybe a breaking change in v4?

[AkihiroSuda]

Yes, the podman network inspect output was changed in v4

• Wire network interface into libpod containers/podman#11322

Podman 3.4.4

$ sudo podman network inspect minikube
[
    {
        "args": {
            "podman_labels": {
                "created_by.minikube.sigs.k8s.io": "true"
            }
        },
        "cniVersion": "0.4.0",
        "name": "minikube",
        "plugins": [
            {
                "bridge": "cni-podman2",
                "hairpinMode": true,
                "ipMasq": true,
                "ipam": {
                    "ranges": [
                        [
                            {
                                "gateway": "192.168.49.1",
                                "subnet": "192.168.49.0/24"
                            }
                        ]
                    ],
                    "routes": [
                        {
                            "dst": "0.0.0.0/0"
                        }
                    ],
                    "type": "host-local"
                },
                "isGateway": true,
                "type": "bridge"
            },
            {
                "capabilities": {
                    "portMappings": true
                },
                "type": "portmap"
            },
            {
                "backend": "",
                "type": "firewall"
            },
            {
                "type": "tuning"
            },
            {
                "capabilities": {
                    "aliases": true
                },
                "domainName": "dns.podman",
                "type": "dnsname"
            }
        ]
    }
]

$ sudo podman network inspect minikube --format '{{range .plugins}}{{if eq .type "bridge"}}{{(index (index .ipam.ranges 0) 0).subnet}},{{(index (index .ipam.ranges 0) 0).gateway}}{{end}}{{end}}'
192.168.49.0/24,192.168.49.1

Podman 4.0.2

$ sudo podman network inspect minikube
[
     {
          "name": "minikube",
          "id": "e0fa249ecbb298366798da5840753b985ebcc6b011a28d037d74e4a925ef52b1",
          "driver": "bridge",
          "network_interface": "podman1",
          "created": "2022-03-30T05:28:49.494426782Z",
          "subnets": [
               {
                    "subnet": "192.168.49.0/24",
                    "gateway": "192.168.49.1"
               }
          ],
          "ipv6_enabled": false,
          "internal": false,
          "dns_enabled": true,
          "labels": {
               "created_by.minikube.sigs.k8s.io": "true"
          },
          "ipam_options": {
               "driver": "host-local"
          }
     }
]

$ sudo podman network inspect minikube --format '{{range .plugins}}{{if eq .type "bridge"}}{{(index (index .ipam.ranges 0) 0).subnet}},{{(index (index .ipam.ranges 0) 0).gateway}}{{end}}{{end}}'
ERRO[0000] Printing inspect output: template: network inspect:1:19: executing "network inspect" at <.plugins>: can't evaluate field plugins in type interface {} 

[AkihiroSuda]

PR

• Podman: fix minikube delete for Podman v4 #13881