docker-machine env xxx hangs forever #1500
Comments
|
I have similar issue with windows 7 OS. here is a gist of the log file. |
|
I am noticing "docker-machine env machinename" hang a lot also, especially when it is in my ~/.profile, causes new terminals to hang and not start 50% of the time approx. Currently just ctrl+c'ing the new tab shell, and spawning another one 2 seconds later tends to do the job. Difficult to reproduce in a new/active terminal, tends to be far more often when spawning a new one. OSX 10.10.4, Docker Machine 0.3.0. These issues were not present in 0.2.x. |
|
I have same issue with strange exit status 255 |
|
I am able to reproduce this issue on MacOSX for the virtualbox driver. The issue typically happens when the route to the private network is missing (the one used by docker cli). Where |
|
I've tried @chantra's solution but didn't do the trick, |
|
Hi all, anyone who is encountering this issue, one of the possible issues at play is that the Docker daemon is not running. Can you please try / paste the output of the following commands (this is for VMs running boot2docker): $ docker-machine -D ssh machinename sudo /etc/init.d/docker restart
...
$ docker-machine -D env machinename
...Thanks! |
|
@nathanleclaire when I try to run that command, it hangs here: Once if finishes (if it ever does) I'll paste what's next. |
|
@inkel Hm, it should run pretty much right away if it's going to succeed, so if it takes more than a few seconds something's wrong. |
|
It just finished with: It took definitively more than a few seconds. I'm running again with |
|
|
If you run : $ /usr/bin/ssh -vvv -o PasswordAuthentication=no -o IdentitiesOnly=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -i /Users/inkel/.docker/machine/machines/dev/id_rsa -p 54973 docker@localhost sudo /etc/init.d/docker restartfrom the CLI on its own, what do you get? |
|
|
I think I've made a mistake with my previous comment, this is the right output: |
|
Mmm... after a few minutes, the following appeared on my screen: |
|
@inkel Odd. Do you have a VPN or any unusual setup as far as your network goes that you can think of? |
|
No, not that I can think of. A few days ago I was still using |
|
@nathanleclaire after talking with my colleague I've found that I was using the following version, 'cause the one in homebrew was giving me this error: He is using the latest one, so I deleted that version, |
|
@inkel Nice, glad you got it working. |
|
I just tried again with the most recent version of docker-machine with no success. I am runing on windows 7 with git shell. All steps and respective outputs are here. The out.log file contains the main steps. Then there is a docker-machine-create.log and a docker-machine-env.log After manually coping the cert files from /var/lib/boot2docker/tls/ to /c/Users/Pedro/.docker/machine/machines/test/ I did another try and the resulting log is docker-machine-env-2.log The vm did start and the docker service was ruining. The interesting thing is that boot2docker works just fine. |
|
This issue is really hard to reproduce... After deleting all vmboxnet_x except vmboxnet0 and vmboxnet1, and rebooting the computer (rather than docker-machine), my issue was gone. |
|
After putting my computer to sleep a couple of times and then trying again I've started to have the same issue. When I had a momento to fully restart it I'll try again and see what happens. |
|
I have the latest versions of all Docker apps. I have gotten the issue on and off for weeks with Docker Machine on Linux Ubuntu building VirtualBox VMs. I just added Docker Swarm and the problem got several times worse (more frequent). I have spent two hours trying to create three swarm machines. If I reboot it usually fixes it, but that is not reasonable. Using docker-machine's --debug you can see where it's hanging as other commenters have pointed out. I see it hang in a few spots with |
|
@garystafford what does your user's ssh config look like? see: #1591 |
|
@chantra's solution worked for me. indeed, the cause was my routes being clobbered by VPN software |
|
I just ran into this as well, @chantra's solution worked for me as well. Cisco AnyConnect 😡 |
|
Also Cisco AnyConnect. I don't suppose you could add a check that recommends fixing the routing table if it's been mangled by software like this? It's VERY frustrating to diagnose; I was still able to ssh into the docker machine, which booted successfully, and had to look at many tickets before I found the discussion in this one. |
This is actually the exact solution I'm considering right now. In many cases, it seems that we can actually successfully finish the creation of the instance and get an IP address, it just happens to not be reachable. |
|
I also have the problem of unreachable docker after sleep. How would you do @chantra's solution in Windows 10? |
|
i experienced this issue also after several sleeps. |
|
This happened to me too when I have docker-machine running and then I get on a VPN, command just hangs forever. |
This just happened to me too: I had docker-machine running, got on VPN, got off VPN, and However I did |
|
I'm having this problem too on Mac OS 10.10.4 with the Cisco AnyConnect VPN.
Tried docker-machine restart default, problem persists. @chantra's solution seems to work for me. On my machine the interface was vboxnet0 (found using ifconfig from the mac terminal) Way easier than rebooting every time I need to go on the VPN. |
|
Not sure I had commented what VPN I was using, but FWIW, I also using cisco On Wed, Sep 23, 2015 at 7:52 PM, Joe McGlynn [email protected]
|
|
FWIW I also was using Cisco AnyConnect VPN, on OS X 10.10.5 I just tried openconnect instead and it seems to avoid the problem. |
|
@ronen , Thanks for the tip about openconnect, I never knew that existed. GoodBye AnyConnect 😀 |
|
@ChrisRut yeah I just learned about openconnect when trying to solve this problem. GoodBye AnyConnect indeed! I found openconnect to be a bit cumbersome to use though, so I threw together a quick wrapper that lets you just type "vpn up" and "vpn down". It's at https://gist.github.com/ronen/7d486adbde5d6bfd2472 if you're interested |
|
As it seems we have multiple issues on similar symtomps, I opened #1934 for those who experience problems on Windows 10 (or maybe other versions) hosts after sleep. In order to separate concerns, I kindly suggest relating this issue's title to VPN network issues. |
|
I get this issue after my host machine has gone to sleep, and I come back to it later. I end up deleting everything from my c:\users\dan.docker\machine folder (other than the cache), then recreate it all again. It's quicker doing that than rebooting the host machine! ;) |
|
Oh, and I also have to kill some processes before doing that. I tend to have 3 'VBoxHeadless.exe' processes, 3 'VBoxNetDHCP.exe', and a 'VirtualBox Interface'. |
|
I woke up my notebook and deleted a docker-machine instance with name "dev". The The stdout log output can be found in this Gist. The strange thing is that the correct output is dumped after the maximum number of retries is reached, see the end of the log. This is the timed stdout output: And it keeps getting stranger... |
|
@mrumpf If you add a trailing slash to the *PROXY options e.g. |
|
I'm getting this right now on Debian 8.2. |
|
Add Jupiter Junos Pulse to the list of VPN software that seems to cause this problem. It strikes me as likely that any VPN would do this, since they all have the same fundamental definition to override "ordinary" routing. Update: by using a "split connection" (an option that my installation allows. by connecting to a different VPN server), I seem to avoid this. |
|
Wait what? I'm not using any kind of VPN or proxy... |
|
VPN is just one of many things that mess with network configuration. More generally, any change in network connections might (and that includes some things you might not think of as "networking"):
The VPN case is noteworthy because it may offer the "split" option, which may avoid flummoxing the Docker connection from this cause. But split VPN isn't going to prevent all those other things from messing things up. |
|
I'm using an old-fashioned RJ45 cable... I don't understand why docker-machine is that sensitive to these events. |
|
I tried rj45 plug/unplug, wifi switching. No issue on my side. |
|
Could there be a need for some processing time? I keep running into this with a shell function that does: When I do these commands one at a time, with my fingers, it goes fine. But when run from the script, the same command sequence works once, than hangs (or takes several minutes). Connecting to @mrumpf , this VM is the second in my |
|
@jrep Ah, in that case, yes, you definitely need to wait a brief interval for the daemon to start up and begin accepting requests. It's why Machine has code to wait for Docker in between our daemon restarts during provisioning. Arguably, we should check for that on |
|
This issue is very long and contains a lot of digressions. If someone continues to encounter similar ones please open a new one at https://github.com/docker/machine/issues/new with detailed information including:
Thanks!
|
and others
The machine is running, and I could use
docker-machine ssh xxxlogin that machine.But
docker-machine env xxxhangs forever. The debugging output said the "host is down". Butdocker-machine lsgave me"NAME ACTIVE DRIVER STATE URL SWARM
dev-test virtualbox Running tcp://192.168.99.100:2376 "
possible related issue: #1168
docker-machine version: v0.3.0
The full info:
Cmd:
docker-machine -D env dev-testOutput:
The text was updated successfully, but these errors were encountered: