If management interface of the VCH is not Layer 2 adjacent to vSphere management endpoints VCH will not work.

[mlh78750]

Configuration of the management gateway for the VCH needs to include networks that should use that gateway.

Currently we have the options:

--management-network 'network 2'
--management-network-gateway 192.168.2.1/24
--management-network-ip 192.168.2.10/24

for setting up a VCH.

The problem is that the external (soon to be renamed public) interface should have the default route. Therefore, we need to allow the customer to specify the network(s) that need to route over the management network/interface of the VCH and through the gateway provided in --management-network-gateway. The way it works in .7.0 is that randomly one of the interfaces and gateways becomes the default route on the VCH. This means that if the external interface gateway is picked as the default route, then all vSphere management traffic that is not L2 adjacent to the VCH management interface will route out of the external interface and for all reasonable security configurations for customers will not reach the target.

Proposal:

Change --management-network-gateway 192.168.2.1/24 to --management-network-gateway 192.168.3.0/24,192.168.128.0/22:192.168.2.1/24. What this would do is cause the VCH to know that all of the network addresses in 192.168.3.0-255 and 192.168.128.0-192.168.131.255 can be reached by sending packets to 192.168.2.1. Which would result in vic-init adding the following routes into the VCH.

Destination     Gateway         Genmask
192.168.3.0    192.168.2.1     255.255.255.0
192.168.128.0  192.168.2.1    255.255.252.0

The default route would remain on the external interface. The specifier would allow 1 to many network spaces separated by commas with a colon and then the gateway with mask.

Workaround for 0.7.0:
Use vic-machine debug to turn on ssh access to the VCH.
Login to the VCH via ssh and ensure the default route is on the external interface.
Add routes as needed to the management network.

[mlh78750]

cc @hmahmood

[mlh78750]

cc: @sflxn as @hmahmood said you saw something similar yesterday.

[mlh78750]

@hickeng: @corrieb just explained that you were thinking of a similar issue with the container networks. Wanted to make sure you saw this.

[vmwarelab]

@mlh78750 to Login to the VCH via ssh and ensure the default route is on the external interface what is the default password for root ?

[stuclem]

@vmwarelab my understanding is that you have to set the root password explicitly when you enable SSH access on the VCH by running vic-machine debug --enable-ssh --rootpw '_password_'. If you just use --enable-ssh without specifying --rootpw, I believe that the password is randomly generated.

@mlh78750 and @hickeng is this assumption correct? Should --enable-ssh require you to also specify either --rootpw or --authorized-key?

[mlh78750]

@stuclem is correct. You will set the password or key with the debug command.

[stuclem]

@mlh78750 in that case, shouldn't we make specifying either of --rootpw or --authorized-key mandatory if you specify --enable-ssh? At the moment you can use --enable-ssh on its own but then you are not able to log in to the VCH because you don't know the random password...

[hickeng]

@stuclem the default password is "password" - I'd highly recommend that people specify a different one.

Until you run the debug command, neither console access nor ssh is enabled so the password is irrelevant.

Higher debug level with vic-machine (I think >2) results in the console login being enabled by default, and the default password is provided for that scenario specifically.

We could modify this so that if you don't specify a password then a randomly generated one is used and printed to the user. That would be trivial to accomplish, but this is intended for interactive debug purposes only - it shouldn't be needed or used in normal operation or deployment once we've identified and addressed our current failure scenarios.

[stuclem]

@hickeng thanks for the explanations. I wasn't requesting a randomly generated password. This what just what I believed happened. Logged #3124 to update the doc per your comments above.

[stuclem]

Just noticed the kind/note flag on this one. Attempted to write this up as follows:

• Virtual container host does not work if management interface is not Layer 2 adjacent to vSphere management endpoints. #3081
  In vSphere Integrated Containers 0.7.0 one of the interfaces and gateways that you configure on the networks is selected randomly to serve as the default route on the virtual container host. However, the external interface should have the default route. As a consequence, if the external interface gateway is selected as the default route, all vSphere management traffic that is not L2 adjacent to the management interface of the virtual container host is routed out of the external interface. In all reasonable security configurations, this traffic will not reach the target.

  Workaround:

  1. Use vic-machine debug to enable SSH access to the virtual container host.
  2. Login to the VCH via SSH
  3. Ensure the default route is on the external interface.
  4. Add routes to the management network as required.

@mlh78750 is this anywhere near accurate?

[mlh78750]

@stuclem That looks good. We might need to give them some more details on how to add the routes.

# ip addr
<list of interfaces>
# route add -net 192.168.2.0 netmask 255.255.255.0 gw 192.168.1.254 dev eth1 

Where eth1 is replaced by the interface that has the management IP in the address list of the first command.
And 192.168.2.0 is replaced with the network address of the network you want to route over the management gateway.
And 255.255.255.0 is replaced with the netmask for the network you want to route over the management gateway.
And 192.168.1.254 is replaced with the ip address of the management gateway.

[mlh78750]

Note that after we fix the 3 vnic limitation #2802 in the VCH that we need this same fix for the client network. I have opened #3125 to track this.

[stuclem]

Thanks @mlh78750, updated RNs accordingly. Removing kind/note.