VMware Desktop Plugin in WSL (maybe some useful info)

[jchappell82]

Vagrant version

vagrant --version = 2.0.3
vmware-desktop plugin (latest as of 3/28/2018- I already uninstalled it so I can't easily check now)

Host operating system

Windows 10 - WSL - version 1709 build 16299.309
Linux flavor - Ubuntu 16.04
VMware Workstation Windows - 12.5

Guest operating system

Ubuntu 16.04

Expected behavior

The VM boots

Actual behavior

The VMware plugin tries to run a hardcoded Windows path to vmrun.exe, rather than allowing either a lookup via $PATH or converting to a WSL supported /mnt/c/path/to/vmware/vmrun.exe path

Steps to reproduce

1. Install vagrant-vmware-desktop plugin in WSL
2. Install the Vagrant VMware Utility in Windows because WSL does not support background services
3. Copy certificates from the Windows utility's path to the appropriate location in the WSL filesystem
4. Attempt a vagrant up --provider=vmware_desktop

It does seem to be able to successfully contact the Utility service, but fails upon attempting to execute vmrun.exe due to the hardcoded Windows-style path.

Some general notes - I totally understand that this isn't officially supported yet, but I thought I'd open this issue in case the information is useful to you during your adding the feature for the next version.

I did also attempt to run the Linux version of the VMware utility in a foreground process from another tab, but that fails with an "unable to locate executables" error (which is almost certainly because it doesn't know how to look at /mnt/c yet)

All that said, this seems to be really close to working. Looking forward to the next release, and if you need any help testing it out, feel free to reach out. Feel free to close this issue as well, as it's just intended to be some hopefully useful info.

[chrisroberts]

Hi there,

Thanks for opening this issue, the info, and the offer to help test out support. I have been spending some time working on this and am close to having it working. Working within the WSL can be challenging when due to a POSIX like environment that's really Windows, and proper path conversions in the correct locations to achieve the expected results. As soon as I have things working as expected I will push a new release.

Again, thanks for all the information, and I hope to have support out very soon.

Cheers!

[StefanScherer]

Hi there,

Today a colleage asked me how to use Vagrant VMware in WSL. He has a brand new business notebook with Windows 10, so he wants to go with WSL. He started to install it, but had troubles.
I tried to help and yes, it's possible to run it. Here are the tweaks we did to make it work.

The following steps describe our environment with Vagrant 2.0.4, Vagrant VMware Desktop plugin 1.0.3 and the Vagrant VMware Utility 1.0.1 and VMware Workstation 14.1.1.

Steps on Windows 10

• uninstall Vagrant on Windows if you have installed it
• install the Vagrant VMware Utility, open an Admin PowerShell terminal

choco install -y vagrant-vmware-utility

(New choco package is still pending in review, otherwise just skip the -version and -pre options)

• create a Vagrant folder in your Windows directory. That's what my colleage has used with the Windows version of Vagrant previously.

mkdir $env:USERPROFILE\.vagrant.d

• create a workspace folder where you want to put your sources and projects.
  This should be a Windows path as you want to edit files with the Windows VSCode / Atom or other editor.

mkdir $env:USERPROFILE\workspace

• here comes the first tweak. Create a symlink in the Windows filesystem to workaround a problem with wrong Unix paths for the vmrun.exe. The plugin calls vmrun.exe with Unix paths like /mnt/c/Users/... and then the vmrun.exe tries to open the file C:\mnt\c\Users\... and fails.

mkdir \mnt\c\Users
mklink /D \mnt\c\Users\<Windows User> \Users\<Windows User>

• install WSL and for example Ubuntu

Steps in Ubuntu WSL

• open a bash terminal

• create a symlink to use the Windows Vagrant folder.

ln -s /mnt/c/Users/$(cmd.exe /c echo %USERNAME% | tr -d '\r')/.vagrant.d/ ~/.vagrant.d

• create a symlink for your sources and projects you want to work with.

ln -s /mnt/c/Users/$(cmd.exe /c echo %USERNAME% | tr -d '\r')/workspace/ ~/workspace

• create a symlink to use the certificates of the Windows Vagrant VMware Utility in WSL.

sudo ln -s /mnt/c/ProgramData/HashiCorp/vagrant-vmware-desktop /opt/vagrant-vmware-desktop

• install Vagrant in WSL

wget https://releases.hashicorp.com/vagrant/2.0.4/vagrant_2.0.4_x86_64.deb
sudo dpkg -i vagrant_2.0.4_x86_64.deb

• install Vagrant VMware Desktop plugin and add your license

vagrant plugin install vagrant-vmware-desktop
vagrant plugin license vagrant-vmware-desktop ./license.lic

• insert following environment in your .bashrc or profile

export VAGRANT_WSL_ENABLE_WINDOWS_ACCESS="1"

• here comes the second tweak: The Vagrant plugin tries to call the absolute Windows path to vmrun.exe and fails ins WSL. We create a script with this exact name (yes, backslashes are valid characters in file names :-) ). A simple symlink is not enough as there is another problem with the Windows output of vmrun.exe list which shows Windows paths that have to be translated back to Linux paths.

#!/bin/bash
if [ "$1" == "list"  ]; then
  "/mnt/c/Program Files (x86)/VMware/VMware Workstation/vmrun.exe" $1 | sed 's/C://' | sed 's/\\/\//g'
elif [ "$1" == "addSharedFolder"  ]; then
  "/mnt/c/Program Files (x86)/VMware/VMware Workstation/vmrun.exe" $1 $2 $3 C:${4//\//\\}
elif [ "$1" == "setSharedFolderState"  ]; then
  "/mnt/c/Program Files (x86)/VMware/VMware Workstation/vmrun.exe" $1 $2 $3 C:${4//\//\\} $5
else
  "/mnt/c/Program Files (x86)/VMware/VMware Workstation/vmrun.exe" $*
fi

Edit: Updated the helper script to fix shared folders as well.

• Put this little bash script into your Linux PATH and make it executable.

sudo vi "/usr/bin/C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe"
chmod +x "/usr/bin/C:\Program Files (x86)\VMware\VMware Workstation\vmrun.exe"

• Now you should be able to run Vagrant in WSL in your workspace folder.

cd ~/workspace
mkdir test
cd test
vagrant init bento/ubuntu-16.04
vagrant up --provider vmware_desktop

This was lot of fun to find out the tweaks. I know this isn't officially supported, but it helped us to get started with our new Windows 10 machines. I hope an upcoming version of the Vagrant Desktop plugin work more smoothly in WSL.

[chrisroberts]

@StefanScherer That is amazing! I'm doing windows work currently (getting hyper-v and the vmware providers working properly in WSL being part of the list) and having the existing release working with those tweaks is fantastic.

[StefanScherer]

Thanks @chrisroberts
But twice during these discoveries I shouted "where are our MacBooks?" 😆
To make it look easy is very hard.

[mikeroySoft]

Interesting...nice work!
Disclaimer: I am the PM for Workstation =)

[mikeroySoft]

If there's anything we can do here I'd be happy to have our engineering teams investigate.

[StefanScherer]

At least this approach running Linux Vagrant in WSL is much much better than my first approach in November last year. I tried to use Windows Vagrant in WSL which then totally failed using vagrant ssh. See PowerShell/Win32-OpenSSH#990 - I tried to put the OpenSSH + WSL teams together, but it seems like it's impossible to make a Windows ssh.exe work in a WSL session.

[WSLUser]

@StefanScherer What Windows build number are you using for those tweaks?

[seal-tf]

@DarthSpock We made the tweaks with Win10 Release 16299.15.amd64fre.rs3_release.170928-1534

[WSLUser]

Interesting. Very cool to know. So I have a strong feeling build 17134 (SCU) and up will require less hacking for it. See the WSL Release Notes for why I believe this to be the case.

[WSLUser]

Also, validation was asked for in microsoft/WSL#2406 (comment) for builds 17627+ that vagrant.exeis working as ssh.exe was confirmed to work as of that build. Granted this is for the Windows version but still be cool to have both with maybe some interop between utilizing AF_Unix interop or some other mechanism. I'm sure some use cases will come up where it would be optimal to have it working from either side. You'll need to be signed up for Skip-Ahead or be on Insiders Fast Ring and wait for release of SCU (that will be when Skip-Ahead and Fast are equal) to test this out.

[StefanScherer]

Thanks @DarthSpock for that info. Our machines are still at Windows 10 1709 for a while.

@chrisroberts I did some more testing with the tweaks. I have problems with the shared folders. Tried it with an bento/ubuntu-18.04 and my StefanScherer/windows_10 boxes. The shared folder will be created from Vagrant, but with the Unix path /mnt/c/Users/.... It seems that VMware Workstation does not really want these unixish pathes. Inside the VM you cannot access the C:\vagrant folder:

When I edit the share manually to either C:\mnt\c\Users\... (with the mklink symlink from one of the tweaksor the real Windows pathC:\Users...` then the VM is able to use the share:

The Vagrant VMware plugin should be improved that it uses the Windows path for the Vagrant shared folder.

A second problem I found is that vagrant rdp does not work inside WSL. The Linux Vagrant wants to run the Linux RDP clients. Vagrant should recognize that it's running in WSL and then use the Windows RDP client. Otherwise I had to run a X11 server as well and struggle with the less featured Linux RDP clients.

General notice: I've moved the whole ~/.vagrant.d folder outside of the WSL world into the Windows world as my information about WSL is that Windows processes shouldn't write into the WSL file system as it can be corrupted in the worst case. As VMware Workstation is reading/writing the VM files I think it's better to set VAGRANT_WSL_ENABLE_WINDOWS_ACCESS=1. Are there other thoughts on this?

[StefanScherer]

Another issue with Vagrant in WSL is that it doesn't recognize other installed providers. A simple vagrant status suggests to install VBoxManage.exe, but I have two other providers vagrant-vmware-desktop and vagrant-vcloud installed:

[WSLUser]

Our machines are still at Windows 10 1709 for a while.

Why don't you spin up a VM with build 17655. It's free as long as you have a license for stable, which you do (I'm also assuming you have Enterprise license).

A second problem I found is that vagrant rdp does not work inside WSL.

So pretty sure you tried this but since I don't see it mentioned, xrdp will allow Windows RDP to work. Just install it in WSL and use the port number it assigns you or change the port number if you want. If there is a way to map xrdp to your vagrant rdp, it should be able to run in Windows RDP. Don't forget to start the xrdp service before running RDP since there's still no autostarting of services unfortunately.

Edit: I just remembered that wouldn't work since you can't start Windows RDP with xrdp which I'm assuming vagrant rdp will want to do. I think linking with xrdp will still be required but also mstsc will also need to be linked.

Another issue with Vagrant in WSL is that it doesn't recognize other installed providers.

I would hazard a guess and say this is due to Windows environmental variables not being fed into Path within WSL. This situation is improved with WSL_ENV in SCU. See this blog post. But this may be completely unrelated. What you could do is try adding the path in Windows to your Path in Ubuntu and see if that works or at least "gets you further along". Might be worth filing an issue over at WSL Github.

[WSLUser]

Are there other thoughts on this?

Linux files should NOT be manipulated by Windows in any way, shape, or form for the time being. I believe there is work being done to improve this scenario but good news you can do the reverse (run Windows or Linux services from Linux as background tasks with 17134.1+)

[StefanScherer]

Thanks @DarthSpock

I've opened PR #9758 to add the Windows RDP client which is the most native RDP client.

Running a newer Windows 10 is not an option as I'm investigating WSL for our Windows 10 laptops. We had Ubuntu Desktop VM's on our Windows 7 machines (Win 7 -> Vagrant VMware -> Ubuntu Desktop) to have a proper dev environment and could do that again on Windows 10.
But working the whole day inside a VM with desktop apps doesn't feel right, so I want to give WSL a chance to work with git without CRLF problems and native Windows editors (atom, vscode, ...) and tools (rdp, ...).

[WSLUser]

Cool, must of had blank moment not thinking of simply adding the code used by Windows Vagrant to the Linux variant.

Running a newer Windows 10 is not an option as I'm investigating WSL for our Windows 10 laptops.

You might be doing yourself an injustice. As much as things improved with FCU, there's so much more features and interoperability between WSL and Windows in SCU, you might be best upgrading from 7 to SCU. Either way, I think you'll find VMs will be needed for very little in your dev environment with WSL if at all.

[StefanScherer]

@DarthSpock I also run the latest Windows 10 1803 in a VM on my MacBook, all good. But on company laptops we suffer from rollout policies that delay such short term updates.

I just don't want to suggest our team to run Windows 10 1709 -> Vagrant VMware -> Windows 10 1803 -> WSL do work there. I could use the Ubuntu Desktop VM instead or just "buy me a MacBook" :-)

[WSLUser]

Yes we all know Iterm2 is a better terminal at this point but with every release, the Windows Console is getting better and better. So for awareness purposes, they are working on ridding themselves of that old UTF-16 code and moving to UTF-8 (a very long and painful endeavor for them with the thousands upon thousands of code relying on UTF-16) and are implementing xterm inside of conhost. I should also mention they have an opensource ColorTool in which you can import all the Iterm2 schemes (because they know people want stuff similar to Iterm2 as it's been mentioned enough times in the Console issue tracker) and other schemes. Also, as a work-around for current limitations of the Console, you can install wsltty which uses Mintty (Cygwin terminal) as a backend. No, you don't need to install Cygwin, the requires files from there and WSLBridge are the only files included.

I just don't want to suggest...

I wouldn't either. Perhaps your org needs to become more Agile. Of course that's assuming good IT management practices are already in place...coughs ITIL, COBIT, Lean Six Sigma, ISO 27001, etc. coughs

[StefanScherer]

I've updated the vmrun helper bash script to fix the shared volume problem.

https://github.com/StefanScherer/dotfiles/blob/c42f54d9f0fa0ee37f699bdfacb2decbb86d0926/bin/vmrun.exe-helper#L4-L7
fixes eg. /mnt/c/Users/... into C:\mnt\c\Users\... and therefore still needs the mklink symlink tweak as well.

[StefanScherer]

Created PR #9759 to allow other providers than VirtualBox for commands like vagrant status.

[StefanScherer]

We currently run into an issue with a second network.

The simple Vagrantfile looks like this and adds a private network with a static IP address:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  config.vm.box = "bento/ubuntu-18.04"
  config.vm.network "private_network", ip: "172.16.32.3"
end

After a vagrant up the vmnet2 network is created.

A vagrant ssh just returns to the WSL prompt after some seconds.

A look at vagrant ssh-config shows that Vagrant wants to use this new network.

Another look at ifconfig in WSL (or ipconfig in PowerShell) shows that the Windows 10 machine does not have a valid IP address for the vmnet2 / eth4.

In the Virtual Network Editor I can see that the vmnet2 does not have DHCP enabled. I think this is missing in the current Vagrant VMware Desktop plugin.
I have to update my setup, but at the moment I'm running Vagrant 2.0.4, vagrant-vmware-desktop (1.0.3) with VMware Workstation 14.1.1.

[seal-tf]

We have enabled DHCP manually in the vmnet2 network. Alternatively we also tried to just set a fixed IP address 172.16.32.1 to the vmnet2 device. A first vagrant up works and also the vagrant ssh.

But after a vagrant destroy -f and another vagrant up we run into another issue:

DEBUG network: Available slots for high-level adapters: #<Set: {1, 2, 3, 4, 5, 6, 7, 8}>
 INFO network: Determining network adapters required for high-level configuration...
 INFO network:  -- Slot 1: hostonly
 INFO network: Determining adapters and compiling network configuration...
 INFO network: Slot 0. Type: nat
DEBUG network: Normalized configuration: {:auto_config=>false, :type=>:dhcp}
DEBUG network: Adapter configuration: {:type=>:nat, :slot=>0}
DEBUG network: Network configuration: {:type=>:dhcp, :auto_config=>false}
 INFO network: Slot 1. Type: hostonly
DEBUG network: Normalized configuration: {:auto_config=>true, :netmask=>"255.255.255.0", :type=>:static, :ip=>"172.16.32.2", :protocol=>"tcp", :id=>"8d113fb8-ff28-4893-ba51-0095c7b8fcc2", :subnet_ip=>"172.16.32.0", :adapter_ip=>"172.16.32.1"}
DEBUG vagrant_utility: request METHOD=GET PATH=/vmnet RESPONSE=200
 INFO network: Found matching vmnet device: vmnet2
 INFO network: Checking for hostonly network collisions...
 INFO subprocess: Starting process: ["/bin/netstat", "-nr"]
 INFO subprocess: Command not in installer, restoring original environment...
DEBUG subprocess: Selecting on IO
DEBUG subprocess: stdout: Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.100.30.1     255.255.255.255 U         0 0          0 eth1
10.100.30.0     0.0.0.0         255.255.255.0   U         0 0          0 eth1
10.100.30.226   0.0.0.0         255.255.255.255 U         0 0          0 eth1
10.100.30.255   0.0.0.0         255.255.255.255 U         0 0          0 eth1
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 eth1
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 eth1
192.168.195.0   0.0.0.0         255.255.255.0   U         0 0          0 eth2
192.168.195.1   0.0.0.0         255.255.255.255 U         0 0          0 eth2
192.168.195.255 0.0.0.0         255.255.255.255 U         0 0          0 eth2
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 eth2
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 eth2
192.168.58.0    0.0.0.0         255.255.255.0   U         0 0          0 eth3
192.168.58.1    0.0.0.0         255.255.255.255 U         0 0          0 eth3
192.168.58.255  0.0.0.0         255.255.255.255 U         0 0          0 eth3
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 eth3
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 eth3
127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo
127.0.0.1       0.0.0.0         255.255.255.255 U         0 0          0 lo
127.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 lo
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 lo
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 lo
255.255.255.255 0.0.0.0         255.255.255.255 U         0 0          0 eth4
224.0.0.0       0.0.0.0         240.0.0.0       U         0 0          0 eth4
0.0.0.0         172.16.32.1     255.255.255.255 U         0 0          0 eth4
172.16.32.255   0.0.0.0         255.255.255.255 U         0 0          0 eth4
172.16.32.0     0.0.0.0         255.255.255.0   U         0 0          0 eth4
172.16.32.1     0.0.0.0         255.255.255.255 U         0 0          0 eth4
DEBUG subprocess: Waiting for process to exit. Remaining to timeout: 32000
DEBUG subprocess: Exit status: 0
ERROR warden: Error occurred: The host only network with the IP '172.16.32.2' would collide with
another device 'eth4'. This means that VMware cannot create
a proper networking device to route to your VM. Please choose
another IP or shut down the existing device.

Seems like the plugin does not recognize the already existing vmnet2 network card.

[StefanScherer]

I've someone is interested how we got the VMware plugin working in WSL right now (with some workarounds) here are my slides how to Setup a Dev environment that feels like $HOME on Windows 10 - DevOps Meetup Bamberg, 2018-06-15

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.