Fix glog parse error

[josedonizetti]

Fixes #4866

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: josedonizetti
To complete the pull request process, please assign medyagh
You can assign the PR to them by writing /assign @medyagh in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[josedonizetti]

@medyagh This fixes the problem on the cmd/minikube/main.go, so minikube binary won't display the errors. Although if I add the pkg/initflag on the tests, the error fork/exec returns. I think we should merge this one, to remove the error from the main binary, while I keep investigating what to do for the tests in a separate pull request.

wdyt?

[medyagh]

@medyagh This fixes the problem on the cmd/minikube/main.go, so minikube binary won't display the errors. Although if I add the pkg/initflag on the tests, the error fork/exec returns. I think we should merge this one, to remove the error from the main binary, while I keep investigating what to do for the tests in a separate pull request.

wdyt?

Thanks for the fix, I am curious does this fix, affects the -- command line args and --alsologtostderr

For example would this command still work.

minikube -p p1 kubextl -- something something
Also
The minikube ssh -- ls -lah

[medyagh]

@josedonizetti I think it doens't work when you pass any arguement,

$ ./out/minikube start --vm-driver=hyperkit
😄  minikube v1.2.0 on darwin (amd64)
🔥  Creating hyperkit VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
E0727 13:05:49.197079    5262 start.go:678] StartHost: create: Error creating machine: Error in driver during machine creation: starting with cmd line: loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes base host=lis: fork/exec /usr/local/bin/hyperkit: invalid argument

💣  Unable to start VM: create: Error creating machine: Error in driver during machine creation: starting with cmd line: loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes base host=lis: fork/exec /usr/local/bin/hyperkit: invalid argument

[medyagh]

one solution would be moving everything to k8s.io/klog, which I recommend, since glog has not been updated in a long time.

I did a regex replace all for klog and everything seemed fine except complaining to add logtostderr.

but the only thing I didnt know how to fix it was logtostderr with klog.

@tstromberg if we move to klog, do you know how to add logtostderr to it ?

[josedonizetti]

@medyagh interesting! Shouldn’t integration tests get this problem?

[medyagh]

@medyagh interesting! Shouldn’t integration tests get this problem?

yeah I am puzzeled by it the weird thing is I can not reprodue it, the only thing is, I had some weird profiles set. I am trying to reproduce it

[medyagh]

@josedonizetti I wonder if we need to add the init flag to the docker-machine-drivers too ... or maybe not use glog there

[medyagh]

@josedonizetti here is full history of ops I did, but I can NOT reproduce it, but as we see in the history I did build the binary using your code. however I did NOT rebuild docker-machine-drivers....

@tstromberg what do you think ? how could I not be able to reproduce the exec.fork again:

Full operations that produced weird exec-fork

clone and build :

medya@ ~/workspace/ppl-minikube (master) $ git checkout --track jose/fix-glog-parse-error
Branch 'fix-glog-parse-error' set up to track remote branch 'fix-glog-parse-error' from 'jose'.
Switched to a new branch 'fix-glog-parse-error'
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ make out/minikube
which go-bindata || GO111MODULE=off GOBIN=/Users/medya/go/bin go get github.com/jteeuwen/go-bindata/...
which go-bindata || GO111MODULE=off GOBIN=/Users/medya/go/bin go get github.com/jteeuwen/go-bindata/...
PATH="/Users/medya/Downloads/google-cloud-sdk/bin:/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/bin:~/go/bin/:/usr/local/git/current/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/go/bin:/usr/local/go/bin:/Users/medya/go/bin" go-bindata -nomemcopy -o pkg/minikube/translate/translations.go -pkg translate translations/...
PATH="/Users/medya/Downloads/google-cloud-sdk/bin:/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/bin:~/go/bin/:/usr/local/git/current/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/go/bin:/usr/local/go/bin:/Users/medya/go/bin" go-bindata -nomemcopy -o pkg/minikube/assets/assets.go -pkg assets deploy/addons/...
gofmt -s -w pkg/minikube/translate/translations.go
gofmt -s -w pkg/minikube/assets/assets.go
GOOS="darwin" GOARCH="amd64" go build -tags "container_image_ostree_stub containers_image_openpgp" -ldflags="-X k8s.io/minikube/pkg/version.version=v1.2.0 -X k8s.io/minikube/pkg/version.isoVersion=v1.2.0 -X k8s.io/minikube/pkg/version.isoPath=minikube/iso -X k8s.io/minikube/pkg/version.gitCommitID="08c86b40108179e6676deb2b4c54717cd0735278"" -a -o out/minikube-darwin-amd64 k8s.io/minikube/cmd/minikube
cp out/minikube-darwin-amd64 out/minikube

Then doing these stuff :

medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube profile list
|-----------------|-----------|----------------|-----------|--------------------|
|     Profile     | VM Driver |     NodeIP     | Node Port | Kubernetes Version |
|-----------------|-----------|----------------|-----------|--------------------|
| TestContainerd  | hyperkit  |                |      8443 | v1.15.0            |
| TestPersistence | hyperkit  | 192.168.64.110 |      8443 | v1.15.0            |
| TestStartStop   | hyperkit  |                |      8443 | v1.15.0            |
| lis             | hyperkit  | 192.168.64.109 |      8443 | v1.15.0            |
| minikube        | hyperkit  | 192.168.64.102 |      8443 | v1.15.0            |
|-----------------|-----------|----------------|-----------|--------------------|
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube delete -p lis
🔥  Deleting "lis" in hyperkit ...
💔  The "lis" cluster has been deleted.
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube delete -p TestStartStop
🔥  Deleting "TestStartStop" in hyperkit ...
💔  The "TestStartStop" cluster has been deleted.
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube delete -p TesttPersistence
🙄  "TesttPersistence" cluster does not exist
💔  The "TesttPersistence" cluster has been deleted.
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube delete -p  TestContainerd
🔥  Deleting "TestContainerd" in hyperkit ...
💔  The "TestContainerd" cluster has been deleted.
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube profile list
|-----------------|-----------|----------------|-----------|--------------------|
|     Profile     | VM Driver |     NodeIP     | Node Port | Kubernetes Version |
|-----------------|-----------|----------------|-----------|--------------------|
| TestPersistence | hyperkit  | 192.168.64.110 |      8443 | v1.15.0            |
| minikube        | hyperkit  | 192.168.64.102 |      8443 | v1.15.0            |
|-----------------|-----------|----------------|-----------|--------------------|
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube delete -p TestPersistence
🔥  Deleting "TestPersistence" in hyperkit ...
💔  The "TestPersistence" cluster has been deleted.
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube profile list
|----------|-----------|----------------|-----------|--------------------|
| Profile  | VM Driver |     NodeIP     | Node Port | Kubernetes Version |
|----------|-----------|----------------|-----------|--------------------|
| minikube | hyperkit  | 192.168.64.102 |      8443 | v1.15.0            |
|----------|-----------|----------------|-----------|--------------------|
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube ssh -- ls

💣  Error getting host: Machine does not exist for api.Exists(lis)

😿  Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉  https://github.com/kubernetes/minikube/issues/new/choose


medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube logs

💣  Error getting config: stat /Users/medya/.minikube/profiles/lis/config.json: no such file or directory

😿  Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉  https://github.com/kubernetes/minikube/issues/new/choose
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube start
😄  minikube v1.2.0 on darwin (amd64)
🔥  Creating virtualbox VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
^C
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube delete
🔥  Deleting "lis" in virtualbox ...
💔  The "lis" cluster has been deleted.
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube ssh

💣  Error getting host: Machine does not exist for api.Exists(lis)

😿  Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉  https://github.com/kubernetes/minikube/issues/new/choose

this is were the exec fork error happens

medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube start --vm-driver=hyperkit
😄  minikube v1.2.0 on darwin (amd64)
🔥  Creating hyperkit VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
E0727 13:05:49.197079    5262 start.go:678] StartHost: create: Error creating machine: Error in driver during machine creation: starting with cmd line: loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes base host=lis: fork/exec /usr/local/bin/hyperkit: invalid argument

💣  Unable to start VM: create: Error creating machine: Error in driver during machine creation: starting with cmd line: loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 systemd.legacy_systemd_cgroup_controller=yes base host=lis: fork/exec /usr/local/bin/hyperkit: invalid argument

😿  Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉  https://github.com/kubernetes/minikube/issues/new/choose

Then after this I could not reproduce it ever again !!!

medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube start 
😄  minikube v1.2.0 on darwin (amd64)


⚠️  Ignoring --vm-driver=virtualbox, as the existing "lis" VM was created using the hyperkit driver.
⚠️  To switch drivers, you may create a new VM using `minikube start -p <name> --vm-driver=virtualbox`
⚠️  Alternatively, you may delete the existing VM using `minikube delete -p lis`


🔄  Restarting existing hyperkit VM for "lis" ...
⌛  Waiting for SSH access ...
🐳  Configuring environment for Kubernetes v1.15.0 on Docker 18.09.6
🔄  Relaunching Kubernetes v1.15.0 using kubeadm ... 
⌛  Verifying: apiserver proxy etcd scheduler controller dns
🏄  Done! kubectl is now configured to use lis
medya@ ~/workspace/ppl-minikube (fix-glog-parse-error) $ ./out/minikube ssh -- ls -lah

[josedonizetti]

@medyagh Any luck reproducing? I've tried it on both OSX and Linux with old driver version and newer just compiled versions, and not able to reproduce it.

[medyagh]

@medyagh Any luck reproducing? I've tried it on both OSX and Linux with old driver version and newer just compiled versions, and not able to reproduce it.

I can not reproduce it, it is one of the greatest mystries of this week ! I really wanna find out why ! because according to my history it did happen !

my only possible hypothesis is , it might be to be related to how we build the docker-machine-drivers (given the recent issue we saw today) #4909

I believe if we find the answer to this mystery we can prevent more unwanted things form happening !