Skip to content
This repository has been archived by the owner on Jan 11, 2023. It is now read-only.

Check outbount connection at the start of CSE #3083

Merged
merged 5 commits into from
Jun 1, 2018
Merged

Check outbount connection at the start of CSE #3083

Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e1fa204
test outbound connection in cse
May 25, 2018
843d363
update timeout
May 25, 2018
85d4339
use nc instead
May 31, 2018
b9241af
fix second check
Jun 1, 2018
2b6d3d1
change timeout numbers
Jun 1, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions parts/k8s/kubernetescustomscript.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ ERR_K8S_RUNNING_TIMEOUT=30 # Timeout waiting for k8s cluster to be healthy
ERR_K8S_DOWNLOAD_TIMEOUT=31 # Timeout waiting for Kubernetes download(s)
ERR_KUBECTL_NOT_FOUND=32 # kubectl client binary not found on local disk
ERR_CNI_DOWNLOAD_TIMEOUT=41 # Timeout waiting for CNI download(s)
ERR_OUTBOUND_CONN_FAIL=50 # Unable to establish outbound connection
ERR_CUSTOM_SEARCH_DOMAINS_FAIL=80 # Unable to configure custom search domains
ERR_APT_DAILY_TIMEOUT=98 # Timeout waiting for apt daily updates
ERR_APT_UPDATE_TIMEOUT=99 # Timeout waiting for apt-get update to complete
Expand Down Expand Up @@ -53,6 +54,10 @@ else
REBOOTREQUIRED=false
fi

function testOutboundConnection() {
retrycmd_if_failure 20 5 20 nc -v 8.8.8.8 53 || retrycmd_if_failure 20 5 20 nc -v 8.8.4.4 53 || exit $ERR_OUTBOUND_CONN_FAIL
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good! Let's do more aggressive timeout and retry. How does 120 1 5 sound? That should give us between ~2 and ~10 min (depending on the type of error) to achieve a single successful response

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good let's try that and get success rate data for a few hours and adjust if needed

}

function waitForCloudInit() {
wait_for_file 900 1 /var/log/azure/cloud-init.complete || exit $ERR_CLOUD_INIT_TIMEOUT
}
Expand Down Expand Up @@ -464,6 +469,8 @@ configAddons() {
fi
}

testOutboundConnection

if [[ $OS == $UBUNTU_OS_NAME ]]; then
echo `date`,`hostname`, apt-get_update_begin>>/opt/m
apt_get_update || exit $ERR_APT_INSTALL_TIMEOUT
Expand Down