VIC endpoint delay or error in releasing explicit port mapping

[corrieb]

User Statement:

As a container developer, if I specify an explicit port mapping for a container and I stop or delete the container, I expect to be able to reuse the same port mapping for a subsequent container.

Details:

This is an inconsistent problem. Sometimes I get a delay when I try to reuse a port mapping and sometimes I get a complete failure to connect. I've seen this issue both on VC and ESX.

Here is an example:

$ docker run -d -p 7781:22 evarga/jenkins-slave
28e630f88e3663432dbb8f22e4abe37e258a9a1b6282157a977b4d20883ec40d
$ ssh [email protected] -p 7781
[email protected]'s password: 
(connected fine)
$ docker ps
CONTAINER ID        IMAGE                  COMMAND               CREATED             STATUS              PORTS                        NAMES
28e630f88e36        evarga/jenkins-slave   "/usr/sbin/sshd -D"   12 minutes ago      Up 12 minutes       10.118.69.198:7781->22/tcp   frosty_bassi
$ docker stop 28e
28e
$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
$ docker rm $(docker ps -a -q)
28e630f88e36
1136d922c984
de2719575cef
$ docker run --name test -d -p 7781:22 evarga/jenkins-slave
1f27183f80c8b4387df3e1846069857f44a3cd78901bce59bd6f9b1135d79114
$ ssh [email protected] -p 7781
ssh: connect to host 10.118.69.198 port 7781: Connection refused
$ ssh [email protected] -p 7781
ssh: connect to host 10.118.69.198 port 7781: Connection refused
$ ssh [email protected] -p 7781
ssh: connect to host 10.118.69.198 port 7781: Connection refused
$ ssh [email protected] -p 7781
ssh: connect to host 10.118.69.198 port 7781: Connection refused
$ ssh [email protected] -p 7781
ssh: connect to host 10.118.69.198 port 7781: Connection refused

I'll attach the logs from this VCH below. This was collected on build rc3-13523

[corrieb]

container-logs (10).zip

[emlin]

finally repeated this problem.
I did docker run and connect jenkins, docker stop, docker run again, and then reconnect jenkins, every time, it works well, although it need sometime to establish the connection right after container is created.
After several times, I got several stopped containers with same port mapping, and one running container could be connected.
Then I deleted one of those stopped containers, this issue happens ssh: connect to host 10.162.52.32 port 7781: Connection refused, although the latest container with correct port mapping still running.

The reason is that we'll unmap portmapping every time while the container is stopped or removed. If the current mapping is not created for this container, it will unmap other containers portmapping by mistake.