none driver: provide solution message when docker not running #10165

Gopikrishna73 · 2021-01-18T14:24:29Z

Steps to reproduce the issue:

1.ubuntu@ip-172-31-45-236:~$ minikube dashboard

The control plane node must be running for this command
- To start a cluster, run: "minikube start"
  ubuntu@ip-172-31-45-236:$ minikube config set driver docker
  ! These changes will take effect upon a minikube delete and then a minikube start
  ubuntu@ip-172-31-45-236:$ minikube start
minikube v1.16.0 on Ubuntu 18.04

! Exiting due to GUEST_DRIVER_MISMATCH: The existing "minikube" cluster was created using the "none" driver, which is incompatible with requested "docker" driver.

Suggestion: Delete the existing 'minikube' cluster using: 'minikube delete', or start the existing 'minikube' cluster using: 'minikube start --driver=none'

ubuntu@ip-172-31-45-236:~$ 'minikube start --driver=none

^C
ubuntu@ip-172-31-45-236:~$ minikube start --driver=none

minikube v1.16.0 on Ubuntu 18.04
Using the none driver based on existing profile
Starting control plane node minikube in cluster minikube
Restarting existing none bare metal machine for "minikube" ...
OS release is Ubuntu 18.04.5 LTS
Preparing Kubernetes v1.20.0 on Docker ...
- kubelet.resolv-conf=/run/systemd/resolve/resolv.conf

X Exiting due to K8S_INSTALL_FAILED: updating control plane: generating kubeadm cfg: getting cgroup driver: docker info --format : exit status 2
stdout:

stderr:
panic: reflect: indirection through nil pointer to embedded struct [recovered]
panic: reflect: indirection through nil pointer to embedded struct

goroutine 1 [running]:
text/template.errRecover(0xc0001d5348)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/text/template/exec.go:166 +0x1b5
panic(0x5575eb4aed00, 0x5575eb7ecd00)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/runtime/panic.go:522 +0x1b9
reflect.Value.FieldByIndex(0x5575eb691460, 0xc0003e07c0, 0x99, 0xc000058dc0, 0x2, 0x2, 0x0, 0x5575eb8b0380, 0x5575eb4aed00)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/reflect/value.go:852 +0x32b
text/template.(*state).evalField(0xc0001d52c8, 0x5575eb691460, 0xc0003e07c0, 0x99, 0x7fff625d3776, 0xc, 0x5575eb86f480, 0xc000177f20, 0xc000590a50, 0x1, ...)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/text/template/exec.go:604 +0xa59
text/template.(*state).evalFieldChain(0xc0001d52c8, 0x5575eb691460, 0xc0003e07c0, 0x99, 0x5575eb691460, 0xc0003e07c0, 0x99, 0x5575eb86f480, 0xc000177f20, 0xc000590a40, ...)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/text/template/exec.go:554 +0x222
text/template.(*state).evalFieldNode(0xc0001d52c8, 0x5575eb691460, 0xc0003e07c0, 0x99, 0xc000177f20, 0xc000590a50, 0x1, 0x1, 0x5575eb563440, 0x5575ecaa3830, ...)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/text/template/exec.go:518 +0x116
text/template.(*state).evalCommand(0xc0001d52c8, 0x5575eb691460, 0xc0003e07c0, 0x99, 0xc000177e30, 0x5575eb563440, 0x5575ecaa3830, 0x99, 0xc0001d50e0, 0x0, ...)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/text/template/exec.go:456 +0x7aa
text/template.(*state).evalPipeline(0xc0001d52c8, 0x5575eb691460, 0xc0003e07c0, 0x99, 0xc00058f5c0, 0x0, 0xc00057f198, 0x0)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/text/template/exec.go:430 +0x11e
text/template.(*state).walk(0xc0001d52c8, 0x5575eb691460, 0xc0003e07c0, 0x99, 0x5575eb86f300, 0xc0000b63f0)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/text/template/exec.go:254 +0x49e
text/template.(*state).walk(0xc0001d52c8, 0x5575eb691460, 0xc0003e07c0, 0x99, 0x5575eb86f540, 0xc000177cb0)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/text/template/exec.go:262 +0x145
text/template.(*Template).execute(0xc0003e0740, 0x5575eb80f2c0, 0xc0002b2e10, 0x5575eb691460, 0xc0003e07c0, 0x0, 0x0)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/text/template/exec.go:217 +0x1ea
text/template.(*Template).Execute(...)
/build/docker.io-IpWXdj/docker.io-19.03.6/go/src/text/template/exec.go:200
github.com/docker/cli/cli/command/system.formatInfo(0x5575eb8a92e0, 0xc00030c0f0, 0x0, 0xc000590a00, 0x1, 0x1, 0xc0003e0700, 0x0, 0x0, 0x0, ...)
/build/docker.io-IpWXdj/docker.io-19.03.6/.gopath/src/github.com/docker/cli/cli/command/system/info.go:478 +0x1f6
github.com/docker/cli/cli/command/system.runInfo(0xc000593b80, 0x5575eb8a92e0, 0xc00030c0f0, 0xc0005903c0, 0x0, 0x0)
/build/docker.io-IpWXdj/docker.io-19.03.6/.gopath/src/github.com/docker/cli/cli/command/system/info.go:85 +0x344
github.com/docker/cli/cli/command/system.NewInfoCommand.func1(0xc000593b80, 0xc0002487e0, 0x0, 0x2, 0x0, 0x0)
/build/docker.io-IpWXdj/docker.io-19.03.6/.gopath/src/github.com/docker/cli/cli/command/system/info.go:52 +0x48
github.com/docker/cli/vendor/github.com/spf13/cobra.(*Command).execute(0xc000593b80, 0xc000358010, 0x2, 0x2, 0xc000593b80, 0xc000358010)
/build/docker.io-IpWXdj/docker.io-19.03.6/.gopath/src/github.com/docker/cli/vendor/github.com/spf13/cobra/command.go:762 +0x467
github.com/docker/cli/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc0003aaf00, 0xc000358000, 0x3, 0x3)
/build/docker.io-IpWXdj/docker.io-19.03.6/.gopath/src/github.com/docker/cli/vendor/github.com/spf13/cobra/command.go:852 +0x2ee
github.com/docker/cli/vendor/github.com/spf13/cobra.(*Command).Execute(...)
/build/docker.io-IpWXdj/docker.io-19.03.6/.gopath/src/github.com/docker/cli/vendor/github.com/spf13/cobra/command.go:800
main.runDocker(0xc00030c0f0, 0x5575eb811680, 0xc0000ca010)
/build/docker.io-IpWXdj/docker.io-19.03.6/.gopath/src/github.com/docker/cli/cmd/docker/docker.go:281 +0x1d5
main.main()
/build/docker.io-IpWXdj/docker.io-19.03.6/.gopath/src/github.com/docker/cli/cmd/docker/docker.go:292 +0xf3

If the above advice does not help, please let us know:
- https://github.com/kubernetes/minikube/issues/new/choose

Full output of failed command:

Full output of minikube start command used, if not already included:

Optional: Full output of minikube logs command:

The text was updated successfully, but these errors were encountered:

afbjorklund · 2021-01-18T14:51:36Z

Looks similar to: #10083 and #10155

For some reason, these trivial Docker commands are failing:

docker version --format {{.Server.Version}}

docker info --format {{.CgroupDriver}}

afbjorklund · 2021-01-18T14:58:23Z

That was easy enough to reproduce. Just stop the docker daemon.

$ sudo systemctl stop docker.socket
$ docker version --format {{.Server.Version}}

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
$ docker info --format {{.CgroupDriver}}
panic: reflect: indirection through nil pointer to embedded struct [recovered]
	panic: reflect: indirection through nil pointer to embedded struct

Workaround is to make sure Docker is running properly, before start.

Gopikrishna73 · 2021-01-19T05:33:49Z

Hi Kubernetes/Minikube, Thank you for your reply and the issue got resolved and it is working now. Once again thank you for your support. Regards, Gopikrishna

…

On Mon, Jan 18, 2021, 8:28 PM Anders Björklund ***@***.***> wrote: That was easy enough to reproduce. Just stop the docker daemon. $ sudo systemctl stop docker.socket $ docker version --format {{.Server.Version}} Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? $ docker info --format {{.CgroupDriver}} panic: reflect: indirection through nil pointer to embedded struct [recovered] panic: reflect: indirection through nil pointer to embedded struct — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10165 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APFHGOKSJ2EJVNPQXVHFJG3S2RECDANCNFSM4WHMDDBA> .

medyagh · 2021-01-27T19:14:18Z

minikube could do a better job checking if Docker is running for "None Driver" and provide a better error, I would accept a PR that fixes this in a way that

"docker info" do not create a panic like this:

X Exiting due to K8S_INSTALL_FAILED: updating control plane: generating kubeadm cfg: getting cgroup driver: docker info --format : exit status 2
stdout:

stderr:
panic: reflect: indirection through nil pointer to embedded struct [recovered]
panic: reflect: indirection through nil pointer to embedded struct

instead it should check if docker is not running exit with better message.

medyagh · 2021-01-29T18:46:55Z

as noted by @hetong07 this is caused by this line

https://github.com/medyagh/minikube/blob/72443005764661ff12dafedee6c0afed4829b379/pkg/minikube/bootstrapper/bsutil/kubeadm.go#L64

	cgroupDriver, err := r.CGroupDriver()
	if err != nil {
		return nil, errors.Wrap(err, "getting cgroup driver")
	}

one way to fix this is, we can parse for the error message and if the docker is not run we can return a ErrorType and if it is None Driver, add solution to user that they need to start docker

hetong07 · 2021-01-31T18:20:14Z

/assign

hetong07 · 2021-01-31T23:14:24Z

We can add a check like this

cgroupDriver, err := r.CGroupDriver()
if err != nil {
	if driver.BareMetal(cc.Driver) && r.Name() == "Docker" && strings.Contains(err.Error(), "panic") {
		return nil, errors.Wrap(err, "Docker daemon is not running with None driver")
	}
	return nil, errors.Wrap(err, "getting cgroup driver")
}

But eventually when it returns at

minikube/pkg/minikube/node/start.go

Lines 299 to 300 in a4ee1e0

    
           if err := bs.UpdateCluster(cfg); err != nil { 
        
           	exit.Error(reason.KubernetesInstallFailed, "Failed to update cluster", err)

Should I use something like below to give user advice of running docker daemon?

minikube/cmd/minikube/cmd/start.go

Lines 265 to 267 in 9d5faff

    
           if err != nil { 
        
           	node.ExitIfFatal(err) 
        
           	exit.Error(reason.GuestStart, "failed to start node", err)

afbjorklund · 2021-02-03T18:43:00Z

we can parse for the error message

You can't really parse the error message, since it panics before even outputting one ? Parsing for "panic" sounds icky.

Since it is the client crashing, it doesn't help if the server is local or remote - so it will affect all types of docker daemons.

afbjorklund · 2021-02-03T18:46:29Z

It did fail earlier, though. Just silently:

Preparing Kubernetes v1.20.0 on Docker ...

So there was an error, running "docker version"

But the current code is ignoring it, maybe it shouldn't ?

// Version retrieves the current version of this runtime
func (r *Docker) Version() (string, error) {
        // Note: the server daemon has to be running, for this call to return successfully
        c := exec.Command("docker", "version", "--format", "{{.Server.Version}}")
        rr, err := r.Runner.RunCmd(c)
        if err != nil {
                return "", err
        }
        return strings.Split(rr.Stdout.String(), "\n")[0], nil
}

func showVersionInfo(k8sVersion string, cr cruntime.Manager) {
        version, _ := cr.Version()

hetong07 · 2021-02-03T19:27:23Z

@afbjorklund I think parse the error info can work. Look at the info:

X Exiting due to K8S_INSTALL_FAILED: updating control plane: generating kubeadm cfg: getting cgroup driver: docker info --format : exit status 2
stdout:

stderr:
panic: reflect: indirection through nil pointer to embedded struct [recovered]
panic: reflect: indirection through nil pointer to embedded struct

The X Exiting due to K8S_INSTALL_FAILED shows the minikube does not crash, the stderr panic... info is actually the stderr message of the docker version command captured by minikube.

afbjorklund added co/none-driver os/linux kind/support Categorizes issue or PR as a support question. labels Jan 18, 2021

afbjorklund added the co/runtime/docker Issues specific to a docker runtime label Jan 18, 2021

medyagh mentioned this issue Jan 27, 2021

minikube start --driver=none #10155

Closed

medyagh added good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. needs-solution-message Issues where where offering a solution for an error would be helpful labels Jan 27, 2021

medyagh changed the title ~~Minikube issue while starting on ubuntu ec2~~ none driver: provide solution message when docker not running Jan 27, 2021

medyagh added kind/bug Categorizes issue or PR as related to a bug. and removed kind/support Categorizes issue or PR as a support question. labels Jan 27, 2021

k8s-ci-robot assigned hetong07 Jan 31, 2021

hetong07 added a commit to hetong07/minikube that referenced this issue Jan 31, 2021

Check error info after 'docker info' command to address kubernetes#10165

adfc5b4

medyagh added the priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. label Feb 3, 2021

This was referenced Feb 3, 2021

Return an ErrorType when 'docker version' command crashes and provide an advice. #10359

Closed

adding new exit code word for when runtime not running #10364

Merged

afbjorklund mentioned this issue Feb 19, 2021

none: docker info: panic: reflect: indirection through nil pointer to embedded struct #10515

Closed

hetong07 added a commit to hetong07/minikube that referenced this issue Feb 19, 2021

Check error info after 'docker info' command to address kubernetes#10165

8834832

afbjorklund mentioned this issue Feb 24, 2021

minikube start issue #10588

Closed

medyagh closed this as completed in #10364 Mar 23, 2021

michaelhenkel pushed a commit to michaelhenkel/minikube that referenced this issue Apr 13, 2021

Check error info after 'docker info' command to address kubernetes#10165

633992a

afbjorklund mentioned this issue May 9, 2021

Exiting due to K8S_INSTALL_FAILED: updating control plane: generating kubeadm cfg: getting cgroup driver: docker info --format : exit status 2 #11323

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

none driver: provide solution message when docker not running #10165

none driver: provide solution message when docker not running #10165

Gopikrishna73 commented Jan 18, 2021

afbjorklund commented Jan 18, 2021

afbjorklund commented Jan 18, 2021 •

edited

Loading

Gopikrishna73 commented Jan 19, 2021 via email

medyagh commented Jan 27, 2021 •

edited

Loading

medyagh commented Jan 29, 2021

hetong07 commented Jan 31, 2021

hetong07 commented Jan 31, 2021

afbjorklund commented Feb 3, 2021

afbjorklund commented Feb 3, 2021 •

edited

Loading

hetong07 commented Feb 3, 2021 •

edited

Loading

none driver: provide solution message when docker not running #10165

none driver: provide solution message when docker not running #10165

Comments

Gopikrishna73 commented Jan 18, 2021

afbjorklund commented Jan 18, 2021

afbjorklund commented Jan 18, 2021 • edited Loading

Gopikrishna73 commented Jan 19, 2021 via email

medyagh commented Jan 27, 2021 • edited Loading

medyagh commented Jan 29, 2021

hetong07 commented Jan 31, 2021

hetong07 commented Jan 31, 2021

afbjorklund commented Feb 3, 2021

afbjorklund commented Feb 3, 2021 • edited Loading

hetong07 commented Feb 3, 2021 • edited Loading

afbjorklund commented Jan 18, 2021 •

edited

Loading

medyagh commented Jan 27, 2021 •

edited

Loading

afbjorklund commented Feb 3, 2021 •

edited

Loading

hetong07 commented Feb 3, 2021 •

edited

Loading