Beacon does not reacquire peers after network disruption

[yorickdowne]

🐞 Bug Report

Description

• Prysm 1.0.0-alpha.27
• Ubuntu 20.04
• Dockerized
• Behind many-to-one NAT
• Beacon ports forwarded on firewall

After network connectivity was lost and restored, Prysm did not reacquire peers. Log excerpt

beacon_1            | time="2020-09-29 23:13:12" level=info msg="Connected to eth1 proof-of-work chain" endpoint="https://goerli.infura.io/v3/USERID" prefix=powchain
beacon_1            | time="2020-09-29 23:13:28" level=info msg="Peer disconnected" activePeers=3 multiAddr="/ip4/159.89.195.24/tcp/9000/p2p/16Uiu2HAkzxm5LRR4agVpFCqfVkuLE6BvGaHbyzZYgY7jsX1WRTiC" prefix=p2p
beacon_1            | time="2020-09-29 23:13:59" level=info msg="Peer disconnected" activePeers=2 multiAddr="/ip4/37.187.150.105/tcp/9000/p2p/16Uiu2HAm298dJNNVgKwF4PmGFnibaURGJedX2GnjSaCXJ2pMxKgE" prefix=p2p
beacon_1            | time="2020-09-29 23:14:11" level=info msg="Peer disconnected" activePeers=1 multiAddr="/ip4/87.123.98.49/tcp/9000/p2p/16Uiu2HAmK6MuJEG8naYrFeCctwv5g3mdG5PhkM3ACP2JYUGJmKoi" prefix=p2p
beacon_1            | time="2020-09-29 23:15:11" level=info msg="Peer disconnected" activePeers=0 multiAddr="/ip4/51.15.70.7/tcp/9500/p2p/16Uiu2HAmTxXKUd1DFdsudodJostmWRpDVj77e48JKCxUdGm1RLaA" prefix=p2p
beacon_1            | time="2020-09-30 00:04:08" level=info msg="Connected to eth1 proof-of-work chain" endpoint="https://goerli.infura.io/v3/USERID" prefix=powchain
beacon_1            | time="2020-09-30 08:06:06" level=info msg="Connected to eth1 proof-of-work chain" endpoint="https://goerli.infura.io/v3/USERID" prefix=powchain

Expected behavior: Prysm beacon will reacquire peers, even when it lost all of them, without a restart of the beacon.

[nisdas]

Potentially related to #7220, will be digging into these issues by tomorrow.