Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: increase stability in K8 copyTo, copyFrom, and execContainer by reducing backpressure #684

Merged
merged 3 commits into from
Oct 11, 2024
Merged

fix: increase stability in K8 copyTo, copyFrom, and execContainer by reducing backpressure #684

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3d8a79f
Increase stability in K8 copyTo, copyFrom, and execContainer by reduc…
jeromy-cannon Oct 11, 2024
4814931
pulled from 00341-deterministic-image
jeromy-cannon Oct 11, 2024
e3acf7b
added filter function for local build path copyTo
jeromy-cannon Oct 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 24 additions & 15 deletions resources/extract-platform.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@
# This script fetch the build.zip file and checksum file from builds.hedera.com and then extract it into HapiApp2 directory
# Usage extract-platform <release-version>
# e.g. extract-platform v0.42.5
set -o pipefail

readonly tag="${1}"
if [ -z "${tag}" ]; then
if [[ -z "${tag}" ]]; then
echo "Release tag is required (e.g. v0.42.5)";
exit 1
fi
Expand All @@ -17,23 +18,31 @@
readonly BUILD_ZIP_URL="${HEDERA_BUILDS_URL}/node/software/${RELEASE_DIR}/build-${tag}.zip"
readonly CHECKSUM_FILE="${HEDERA_USER_HOME_DIR}/build-${tag}.sha384"
readonly CHECKSUM_URL="${HEDERA_BUILDS_URL}/node/software/${RELEASE_DIR}/build-${tag}.sha384"
readonly LOG_FILE="${HAPI_DIR}/output/extract-platform.log"
echo "extract-platform.sh: begin................................" | tee -a ${LOG_FILE}

# download
echo "Downloading ${BUILD_ZIP_URL}"
[ -f "${BUILD_ZIP_FILE}" ] || curl -sSf "${BUILD_ZIP_URL}" -o "${BUILD_ZIP_FILE}"
[ $? == 0 ] || exit 1
echo "Downloading ${CHECKSUM_URL}"
[ -f "${CHECKSUM_FILE}" ] || curl -sSf "${CHECKSUM_URL}" -o "${CHECKSUM_FILE}"
[ $? == 0 ] || exit 1
readonly sum="$(openssl dgst -sha384 "${BUILD_ZIP_FILE}" | awk '{print $2}')"
readonly expected_sum="$(awk '{print $1}' < "${CHECKSUM_FILE}")"
if [ "${sum}" != "${expected_sum}" ]; then
echo "SHA sum of ${BUILD_ZIP_FILE} does not match. Aborting."
echo "Downloading ${BUILD_ZIP_URL}" | tee -a ${LOG_FILE}
[[ -f "${BUILD_ZIP_FILE}" ]] || curl -sSf "${BUILD_ZIP_URL}" -o "${BUILD_ZIP_FILE}" | tee -a ${LOG_FILE}
[[ $? -ne 0 ]] && exit 1
echo "Downloading ${CHECKSUM_URL}" | tee -a ${LOG_FILE}
[[ -f "${CHECKSUM_FILE}" ]] || curl -sSf "${CHECKSUM_URL}" -o "${CHECKSUM_FILE}" | tee -a ${LOG_FILE}
[[ $? -ne 0 ]] && exit 1
cd ${HEDERA_USER_HOME_DIR}
sha384sum -c ${CHECKSUM_FILE} | tee -a ${LOG_FILE}

Check warning on line 32 in resources/extract-platform.sh

View check run for this annotation

Codacy Production / Codacy Static Code Analysis

resources/extract-platform.sh#L32 

Double quote to prevent globbing and word splitting.
if [[ $? -ne 0 ]]; then
echo "SHA sum of ${BUILD_ZIP_FILE} does not match. Aborting." | tee -a ${LOG_FILE}
exit 1
fi

# extract
echo "Extracting ${BUILD_ZIP_FILE}"
[[ -d "${HAPI_DIR}" ]] || mkdir -p "${HAPI_DIR}"
cd "${HAPI_DIR}" && jar xvf "${BUILD_ZIP_FILE}"
[ $? == 0 ] || exit 1
echo "Extracting ${BUILD_ZIP_FILE}" | tee -a ${LOG_FILE}
[[ -d "${HAPI_DIR}" ]] || mkdir -p "${HAPI_DIR}" | tee -a ${LOG_FILE}
echo "HAPI_DIR=${HAPI_DIR}" | tee -a ${LOG_FILE}
cd ${HAPI_DIR}
pwd | tee -a ${LOG_FILE}
ls -al | tee -a ${LOG_FILE}
jar xvf "${BUILD_ZIP_FILE}" | tee -a ${LOG_FILE}
[[ $? -ne 0 ]] && echo "Failure occurred during decompress" && exit 1
echo "................................end: extract-platform.sh" | tee -a ${LOG_FILE}
exit 0
9 changes: 7 additions & 2 deletions src/commands/node.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -820,8 +820,13 @@ export class NodeCommand extends BaseCommand {
subTasks.push({
title: `Copy local build to Node: ${chalk.yellow(nodeAlias)} from ${localDataLibBuildPath}`,
task: async () => {
this.logger.debug(`Copying build files to pod: ${podName} from ${localDataLibBuildPath}`)
await self.k8.copyTo(podName, constants.ROOT_CONTAINER, localDataLibBuildPath, `${constants.HEDERA_HAPI_PATH}`)
// filter the data/config and data/keys to avoid failures due to config and secret mounts
const filterFunction = (path, stat) => {
return !(path.includes('data/keys') || path.includes(
'data/config'))
}
await self.k8.copyTo(podName, constants.ROOT_CONTAINER, localDataLibBuildPath,
`${constants.HEDERA_HAPI_PATH}`, filterFunction)
const testJsonFiles = self.configManager.getFlag(flags.appConfig).split(',')
for (const jsonFile of testJsonFiles) {
if (fs.existsSync(jsonFile)) {
Expand Down
Loading
Loading