alternative boot2docker ISO image (minikube-iso) support #135

s-urbaniak · 2016-09-20T12:03:58Z

Hi,

I am building an alternative boot2docker image called minikube-iso https://github.com/coreos/minikube-iso for the https://github.com/kubernetes/minikube project which include systemd, as well as the rkt container runtime. The ISO is built using http://buildroot.org.

When starting using the following command:

$ minikube start --show-libmachine-logs=true --v=7 --logtostderr=true --iso-url=https://github.com/coreos/minikube-iso/releases/download/xhyve/minikube-xhyve.iso --vm-driver=xhyve
I0920 13:37:51.313538   78441 notify.go:103] Checking for updates...
Starting local Kubernetes cluster...
Found binary path at /usr/local/bin/docker-machine-driver-xhyve
Launching plugin server for driver xhyve
Plugin server listening at address 127.0.0.1:65105
() DBG | operation not supported by device
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetMachineName
(minikube) Calling .DriverName
Creating CA: /Users/sur/.minikube/certs/ca.pem
Creating client certificate: /Users/sur/.minikube/certs/cert.pem
Running pre-create checks...
(minikube) Calling .PreCreateCheck
(minikube) DBG | ===== Docker Machine xhyve Driver Version 0.2.3 (Homebrew) =====
(minikube) DBG | 
(minikube) DBG | executing: /usr/local/bin/VBoxManage -v
(minikube) DBG | STDOUT: 5.0.22r108108
(minikube) DBG | 
(minikube) DBG | STDERR: 
(minikube) Calling .GetConfigRaw
Creating machine...
(minikube) Calling .Create
(minikube) Downloading /Users/sur/.minikube/cache/boot2docker.iso from file:///Users/sur/.minikube/cache/iso/minikube-xhyve.iso...
(minikube) Creating VM...
(minikube) DBG | Mounting boot2docker.iso
(minikube) Extracting vmlinuz64 and initrd.img from boot2docker.iso...
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil attach /Users/sur/.minikube/machines/minikube/boot2docker.iso -mountpoint /Users/sur/.minikube/machines
/minikube/b2d-image] []  <nil> 0xc820026010 0xc820026018 [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>} attach /Users/sur/.minikube/machines/minikube/boot
2docker.iso -mountpoint /Users/sur/.minikube/machines/minikube/b2d-image
(minikube) /dev/disk9                                           /Users/sur/.minikube/machines/minikube/b2d-image
(minikube) DBG | Extracting vmlinuz64 into /Users/sur/.minikube/machines/minikube
(minikube) DBG | Extracting initrd.img into /Users/sur/.minikube/machines/minikube
(minikube) DBG | Unmounting boot2docker.iso
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil detach /Users/sur/.minikube/machines/minikube/b2d-image] []  <nil> 0xc820026010 0xc820026018 [] <nil> <
nil> <nil> <nil> false [] [] [] [] <nil>} detach /Users/sur/.minikube/machines/minikube/b2d-image
(minikube) "disk9" unmounted.
(minikube) "disk9" ejected.
(minikube) Generating 20000MB disk image...
(minikube) DBG | executing: &{/usr/bin/hdiutil [hdiutil create -megabytes 20000 -type SPARSEBUNDLE /Users/sur/.minikube/machines/minikube/root-volume] []  <nil
> 0xc820026010 0xc820026018 [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>} create -megabytes 20000 -type SPARSEBUNDLE /Users/sur/.minikube/machines/miniku
be/root-volume
(minikube) created: /Users/sur/.minikube/machines/minikube/root-volume.sparsebundle
(minikube) Creating SSH key...
(minikube) Fix file permission...
(minikube) DBG | /Users/sur/.minikube/machines/minikube/boot2docker.iso
(minikube) DBG | /Users/sur/.minikube/machines/minikube/config.json
(minikube) DBG | /Users/sur/.minikube/machines/minikube/id_rsa
(minikube) DBG | /Users/sur/.minikube/machines/minikube/id_rsa.pub
(minikube) DBG | /Users/sur/.minikube/machines/minikube/initrd.img
(minikube) DBG | /Users/sur/.minikube/machines/minikube/root-volume.sparsebundle
(minikube) DBG | /Users/sur/.minikube/machines/minikube/vmlinuz64
(minikube) Generate UUID...
(minikube) DBG | Generated UUID: 3FDDB05E-2DC8-40BB-AB35-F577EA3A43D2
(minikube) Convert UUID to MAC address...
(minikube) DBG | Converted MAC address: 7a:24:5a:36:cf:32
(minikube) Starting minikube...
(minikube) DBG | [xhyve -A -U 3FDDB05E-2DC8-40BB-AB35-F577EA3A43D2 -c 1 -m 1024M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3,ahci-cd,/Us
ers/sur/.minikube/machines/minikube/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk9 -f kexec,/Users/sur/.minikube/machines/minikube/vmlinuz64,/Users/sur/.minikube/m
achines/minikube/initrd.img,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 base host=minikube -F /Users/sur/.minikube
/machines/minikube/minikube.pid -s 5,virtio-9p,host=/Users]
(minikube) Waiting for VM to come online...
(minikube) DBG | 
(minikube) DBG | Not there yet 1/60, error: IP not found for MAC 7a:24:5a:36:cf:32 in DHCP leases
(minikube) DBG | operation not supported by device
(minikube) Waiting on a pseudo-terminal to be ready... done
(minikube) Hook up your terminal emulator to /dev/ttys005 in order to connect to your VM
(minikube) DBG | vm exit[0]
(minikube) DBG |        reason          VMX
(minikube) Hook up a terminal emulator to /dev/ttys005 in order to access your VM
(minikube) DBG |        rip             0xffffffff81908cdd
(minikube) DBG |        inst_length     5
(minikube) DBG |        status          0
(minikube) DBG |        exit_reason     2
(minikube) DBG |        qualification   0x0000000000000000
(minikube) DBG |        inst_type               0
(minikube) DBG |        inst_error              0
(minikube) DBG | signal: abort trap &{0xc82000e140} &{0xc82000e160}
(minikube) DBG | 
(minikube) DBG | Not there yet 2/60, error: IP not found for MAC 7a:24:5a:36:cf:32 in DHCP leases
(minikube) DBG | 
(minikube) DBG | Not there yet 3/60, error: IP not found for MAC 7a:24:5a:36:cf:32 in DHCP leases
...

Especially the vm exit[0] apparently indicates something, but I am not able to reason about the output. I am sorry, that this lies outside of the "regular" boot2docker ISO, but any help/hint would be helpful. Am i missing some kernel module maybe?

Here is the kernel configuration: https://github.com/s-urbaniak/minikube-iso/blob/xhyve-support/board/coreos/minikube/linux-4.7_defconfig

PS: Creating a minikube instance using minikube start --vm-driver=xhyve works.

Thanks for any hints!

The text was updated successfully, but these errors were encountered:

zchee · 2016-09-20T12:13:30Z

@s-urbaniak Hmm... ok.
I'm do not yet understand that's error meaning, but I remember seeing (recognize? look familiar? sorry I'm not good at English ) that error.
Please wait a moment...

(If I remember right, this error also occurs fuchsia-mirror/magenta's forked qemu. I will investigate.)

zchee · 2016-09-20T12:15:08Z

@s-urbaniak Ah, remembered. fuchsia-mirror/magenta also uses buildroot kernel.
If you have time, try run the other tiny OS?
(but, I will continue the investigate)

zchee · 2016-09-20T12:21:26Z

@s-urbaniak Also, If you have time, could you try launch that ISO uses original xhyve or hyperkit?

If both of failed, docker-machine-driver-xhyve also not supported. Needs C languages level hack(or workaround?).

Edit: If you not familiar with xhyve or hyperkit, I will test it.

s-urbaniak · 2016-09-20T14:37:54Z

@zchee not too familiar to be honest, since I am actually a Linux user only ;-) If you have spare cycles to give it a shot with xhyve, that would be great :-)

zchee · 2016-09-20T15:43:30Z

@s-urbaniak OK, np!
So that question for minikube/minishift or other kubenetes?

Anyway, I will investigate later :)

s-urbaniak · 2016-09-20T15:51:35Z

@zchee this is for minikube :-)

zchee · 2016-09-20T15:52:33Z

@s-urbaniak ok :)

zchee · 2016-09-20T15:56:00Z

@s-urbaniak Sorry, reference for me. What kubenetes team's goal?
Use rkt?

s-urbaniak · 2016-09-20T16:04:59Z

@zchee Kubernetes as of 1.3 also supports rkt as an additional container runtime next to Docker. Furthermore there is a container runtime interface specification out there [1] which abstracts container runtime support inside Kubernetes even more.

[1] https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/container-runtime-interface-v1.md

zchee · 2016-09-20T16:14:35Z

@s-urbaniak Ah, sorry. My question is bad... :(

In other words, What I should be run any iso?
If want to run the buildroot iso, ok, I will debug it use buildroot iso.
But If you want to run other iso, It might be wasted.

So, What do you want to run the iso file instead of boot2docker.iso?

I cannot use English well, so please forgive me if I say something rude.
Edit: or, sorry if I'm misunderstood.

s-urbaniak · 2016-09-20T16:20:33Z

@zchee no, problem, I appreciate the help :-)

To try out the ISO, simply execute:

$ minikube start \
  --show-libmachine-logs=true --v=7 --logtostderr=true \
  --iso-url=https://github.com/coreos/minikube-iso/releases/download/xhyve/minikube-xhyve.iso \
  --vm-driver=xhyve

Note: you must have minikube 0.10.0 (https://github.com/kubernetes/minikube/releases/tag/v0.10.0).

zchee · 2016-09-20T16:23:32Z

@s-urbaniak OK thanks. will try it.

BTW, I've also joined kubenetes Slack channel. Could you help in there?
If ok, #minikube channel?

s-urbaniak · 2016-09-20T16:35:23Z

@zchee sure thing!

zchee · 2016-09-23T16:15:53Z

Current state: https://github.com/coreos/minikube-iso/issues/17

dlorenc · 2016-10-29T19:35:55Z

So with the two PRs I just sent: #144 and #143 I got this to boot with the new ISO!

$ minikube start --iso-url=http://storage.googleapis.com/minikube/iso/buildroot/minikube-v0.0.6.iso --vm-driver=xhyve --show-libmachine-logs --v=100

Unfortunately this doesn't work with the old ISO though. I took a look and the xhyve commands run seem the same, but I might be missing something.

@zchee is there anything else missing in this branch to work with both ISOs?

dlorenc · 2016-10-30T02:36:00Z

Nevermind, I'm not sure what happened but my PR #144 works with both ISOs :)

zchee · 2016-11-03T14:24:39Z

xhyve driver already support minikube iso.
But related issue now discuss on #147.
I'll close this issue thread after the finished that discuss.

zchee · 2016-11-06T21:15:58Z

#147 has been merged, and I was released v0.3.0.
Close.

psinha260689 · 2017-10-26T19:40:38Z

in boot2doecker when i start minikube , they ask minikube login and minikube password. what are the default values ?

zchee added bug feature help wanted labels Sep 20, 2016

hferentschik mentioned this issue Nov 3, 2016

Fix #146 Make /boot configurable to allow different live iso to work with driver #147

Merged

zchee closed this as completed Nov 6, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

alternative boot2docker ISO image (minikube-iso) support #135

alternative boot2docker ISO image (minikube-iso) support #135

s-urbaniak commented Sep 20, 2016

zchee commented Sep 20, 2016

zchee commented Sep 20, 2016

zchee commented Sep 20, 2016 •

edited

Loading

s-urbaniak commented Sep 20, 2016

zchee commented Sep 20, 2016 •

edited

Loading

s-urbaniak commented Sep 20, 2016

zchee commented Sep 20, 2016

zchee commented Sep 20, 2016

s-urbaniak commented Sep 20, 2016

zchee commented Sep 20, 2016 •

edited

Loading

s-urbaniak commented Sep 20, 2016

zchee commented Sep 20, 2016

s-urbaniak commented Sep 20, 2016

zchee commented Sep 23, 2016

dlorenc commented Oct 29, 2016

dlorenc commented Oct 30, 2016

zchee commented Nov 3, 2016

zchee commented Nov 6, 2016

psinha260689 commented Oct 26, 2017

alternative boot2docker ISO image (minikube-iso) support #135

alternative boot2docker ISO image (minikube-iso) support #135

Comments

s-urbaniak commented Sep 20, 2016

zchee commented Sep 20, 2016

zchee commented Sep 20, 2016

zchee commented Sep 20, 2016 • edited Loading

s-urbaniak commented Sep 20, 2016

zchee commented Sep 20, 2016 • edited Loading

s-urbaniak commented Sep 20, 2016

zchee commented Sep 20, 2016

zchee commented Sep 20, 2016

s-urbaniak commented Sep 20, 2016

zchee commented Sep 20, 2016 • edited Loading

s-urbaniak commented Sep 20, 2016

zchee commented Sep 20, 2016

s-urbaniak commented Sep 20, 2016

zchee commented Sep 23, 2016

dlorenc commented Oct 29, 2016

dlorenc commented Oct 30, 2016

zchee commented Nov 3, 2016

zchee commented Nov 6, 2016

psinha260689 commented Oct 26, 2017

zchee commented Sep 20, 2016 •

edited

Loading

zchee commented Sep 20, 2016 •

edited

Loading

zchee commented Sep 20, 2016 •

edited

Loading