From b3a6eee59e61a3124f177627f65b930ce1356f2b Mon Sep 17 00:00:00 2001 From: Jasdeep Singh Date: Wed, 26 Jan 2022 21:50:27 +0530 Subject: [PATCH] Refactor: Docker Compose Easy Setup (#250) * Refactor: Docker Compose Easy Setup Signed-off-by: jay-dee7 * Changed action to reflect the updates * Add: long-syntax for ports in docker-compose config Signed-off-by: jay-dee7 * File structure and workflows * Switched permissions * Copy as root then own it * Added permissions * Created localnet setup extracted from workflow * Removed volume from Dockerfile * Switched directory to match changes * Fixed to match new image settings * External laddr availability * Switched test port to avoid clashes * Bypass defaults * Removed excess * Removed moniker name in container env vars Signed-off-by: jay-dee7 * Fixed supervisor config to match dirs * Removed unwanted quotes * Switched to reflect main * Single Dockerfile for mainnet & testnet * Adjusted to old dir * Removed excess * Upgraded testnet version beforehand * Added whole home dir volume * Multiple driver options & fixes * Testnet ref typo fix * Consistent across mainnet & testnet * Moved environment variables to host * Move single image localnet to tests * Fix naming of build tools * Fx liinks in workflows * Move docker localnet to tests * Fix documentation * Fix case * Fix broken links * Allow 0, 403 status codes for link checker * Fix broken links * Fix file names * Fx links, dockerfile * Fix test pipeline * Fix test pipeline * Fix test * Revert volume names * Fix corner case where host ip is invalid at first launch * Optimize docker file * Corrected spelling in filename * Fix script name in the lint pipeline * Typo fixes in GitHub Actions release workflow * Cosmetic switch to put exposed ports in same order as in config files * Swapping parameter order around in node-runner.sh * Added explanatory comments to Docker Compose file * Added extended descriptions for inside-container configuration file for mainnet * Update Dockerfile * Modified configuration for volumes in Docker Compose file * Extended descriptions in host environment variables * Update .Dockerignore * Update release.yml * Update .gitignore * Changed node-runner script to copy genesis and seed config data from Docker Compose config locations * Optimise Dockerfile by removing unnecessary requirements - Removed unused apt-get install packages in Docker build stages - Added comments for readability * Fixed errors due to incorrect commenting style in Compose environment file * Fixed incorrect variable names in Docker Compose file * Updated Docker Compose files and moved directories * Changed host and container environment files * Update node-runner.sh * Bumped version to v0.3.5 * Updated Dockerfile release compose * Moved production build stage to separate file * Renaming host.env to docker-compose.env * Renaming node-runner.sh to entrypoint.sh * Added build stage to Docker Compose file * Created separate stage Dockerfile for cheqd-node release images * Fix container's parameters Signed-off-by: Andrew Nikitin * Fix cheqd-node image building Signed-off-by: Andrew Nikitin * Rename job name Signed-off-by: Andrew Nikitin * Fix yaml errors Signed-off-by: Andrew Nikitin * Change Dockerfile to multi-stage build * Made build environment variable optional * Commented out build section in Docker Compose file * Delete Dockerfile.release * Debugging docker-compose Co-authored-by: Andrew Nikitin * Debugging docker-compose * Fix image building * Change entrypoint for cheqd_noded_docker cmd Signed-off-by: Andrew Nikitin * Make 77 for all dirs Signed-off-by: Andrew Nikitin * Add UID and GID for image buildingI Signed-off-by: Andrew Nikitin * Change params passing for UID and GID Signed-off-by: Andrew Nikitin * Debug Signed-off-by: Andrew Nikitin * Change entrypoints for tests Signed-off-by: Andrew Nikitin * Update Dockerfile * Update docker-compose.yml * Revert "Update docker-compose.yml" This reverts commit d727e9da7e7b993ad6fb02cc56a5e566c6087296. Co-authored-by: Tasos <50984242+Eengineer1@users.noreply.github.com> Co-authored-by: Alexandr Kolesov Co-authored-by: Ankur Banerjee Co-authored-by: Andrew Nikitin Signed-off-by: Andrew Nikitin --- .Dockerignore | 29 +++++-- .github/scripts/ensure_set_euox_pipefail.sh | 23 ++++++ .github/workflows/lint.yml | 63 +++++++++++++++ .github/workflows/markdown-ci/mlc_config.json | 4 +- .github/workflows/node-ci.yml | 45 ++++++----- .github/workflows/release.yml | 31 +++---- .gitignore | 14 ++-- SUMMARY.md | 6 +- .../build_deb.sh => build-tools/build-deb.sh | 0 .../build_tar.sh => build-tools/build-tar.sh | 0 {build_tools => build-tools}/common.sh | 0 {build_tools => build-tools}/postinst | 2 +- {build_tools => build-tools}/postremove | 0 docker/{cheqd_node => }/Dockerfile | 54 ++++++++----- docker/persistent-chains/container.env | 55 +++++++++++++ docker/persistent-chains/docker-compose.env | 76 +++++++++++++++++ docker/persistent-chains/docker-compose.yml | 81 +++++++++++++++++++ .../entrypoint.sh} | 45 ++++++----- docs/build-and-networks/README.md | 18 +++-- ...ose.md => local-docker-compose-network.md} | 2 +- .../local-docker-network.md | 4 +- ...nd-networks.md => manual-network-setup.md} | 2 +- docs/setup-and-configure/README.md | 2 +- docs/setup-and-configure/binary-install.md | 2 +- docs/setup-and-configure/docker-install.md | 2 +- .../add-observer.sh} | 2 +- .../docker-compose.yml | 18 +++-- .../gen-node-configs.sh} | 3 +- .../promote-validator.sh} | 0 .../run-docker.sh} | 3 +- .../wait.sh | 0 .../networks/docker-localnet}/Dockerfile | 11 +-- .../docker-localnet/gen-node-configs-new.sh | 0 .../docker-localnet/gen-node-configs.sh | 0 .../docker-localnet}/supervisord.conf | 2 +- tests/networks/docker_compose.md | 67 --------------- tests/networks/local_node.md | 48 ----------- tests/networks/local_node/gen_node_config.sh | 80 ------------------ .../wait-for-chain.sh} | 0 39 files changed, 479 insertions(+), 315 deletions(-) create mode 100644 .github/scripts/ensure_set_euox_pipefail.sh create mode 100644 .github/workflows/lint.yml rename build_tools/build_deb.sh => build-tools/build-deb.sh (100%) rename build_tools/build_tar.sh => build-tools/build-tar.sh (100%) rename {build_tools => build-tools}/common.sh (100%) rename {build_tools => build-tools}/postinst (99%) rename {build_tools => build-tools}/postremove (100%) rename docker/{cheqd_node => }/Dockerfile (52%) create mode 100644 docker/persistent-chains/container.env create mode 100644 docker/persistent-chains/docker-compose.env create mode 100644 docker/persistent-chains/docker-compose.yml rename docker/{cheqd_node/node-runner.sh => persistent-chains/entrypoint.sh} (61%) rename docs/build-and-networks/{docker-compose.md => local-docker-compose-network.md} (94%) rename docs/build-and-networks/{build-and-networks.md => manual-network-setup.md} (99%) rename tests/networks/{docker_compose/add_observer.sh => docker-compose-localnet/add-observer.sh} (93%) rename tests/networks/{docker_compose => docker-compose-localnet}/docker-compose.yml (75%) rename tests/networks/{docker_compose/gen_node_configs.sh => docker-compose-localnet/gen-node-configs.sh} (98%) rename tests/networks/{docker_compose/promote_validator.sh => docker-compose-localnet/promote-validator.sh} (100%) rename tests/networks/{docker_compose/run_docker.sh => docker-compose-localnet/run-docker.sh} (88%) rename tests/networks/{docker_compose => docker-compose-localnet}/wait.sh (100%) rename {docker/single_image_testnet => tests/networks/docker-localnet}/Dockerfile (74%) rename docker/single_image_testnet/gen_node_configs_new.sh => tests/networks/docker-localnet/gen-node-configs-new.sh (100%) mode change 100755 => 100644 rename docker/single_image_testnet/gen_node_configs.sh => tests/networks/docker-localnet/gen-node-configs.sh (100%) mode change 100755 => 100644 rename {docker/single_image_testnet => tests/networks/docker-localnet}/supervisord.conf (96%) delete mode 100644 tests/networks/docker_compose.md delete mode 100644 tests/networks/local_node.md delete mode 100755 tests/networks/local_node/gen_node_config.sh rename tests/networks/{wait_for_chain.sh => tools/wait-for-chain.sh} (100%) diff --git a/.Dockerignore b/.Dockerignore index e90d0ad1b..3350a5acb 100644 --- a/.Dockerignore +++ b/.Dockerignore @@ -1,8 +1,25 @@ -ci -.devcontainer +### APP-SPECIFIC .github +*.md +build-tools/* +docs/* +architecture/* +persistent_chains/* + + +### GENERAL EXCLUSIONS ### + +### macOS ### +# General +.DS_Store +.AppleDouble +.LSOverride + +# IDE Exclusions +.devcontainer +.vscode +.vscode-upload.json .idea -vue/node_modules -vue/src/store/generated -vue/dist -secret.yml + +# Python +.pytest_cache diff --git a/.github/scripts/ensure_set_euox_pipefail.sh b/.github/scripts/ensure_set_euox_pipefail.sh new file mode 100644 index 000000000..7cfdfc2a7 --- /dev/null +++ b/.github/scripts/ensure_set_euox_pipefail.sh @@ -0,0 +1,23 @@ +#!/bin/bash + +# Ensures that all bash scripts in the repository use `set -euox pipefail` or `set -euo pipefail` statement at the beginning. + +set -euo pipefail + +INVALID_FILES_FOUND=0 + +for BASH_SCRIPT in $(find . -type f -name "*.sh") +do + if ( ! grep -q "set -euo pipefail" "${BASH_SCRIPT}" ) && ( ! grep -q "set -euox pipefail" "${BASH_SCRIPT}" ) + then + echo "${BASH_SCRIPT}" + INVALID_FILES_FOUND=1 + fi +done + +if [[ INVALID_FILES_FOUND -eq 1 ]] +then + echo "" + echo "The bash scripts above must include either 'set -euo pipefail' or 'set -euox pipefail." + exit 1 +fi diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml new file mode 100644 index 000000000..c7a619f45 --- /dev/null +++ b/.github/workflows/lint.yml @@ -0,0 +1,63 @@ +name: Lint + +on: [push] + +jobs: + + # We can't use VALIDATE_GO from super linter because of this issue: + # https://github.com/github/super-linter/issues/143 + go-lint: + name: Lint go + runs-on: ubuntu-20.04 + steps: + - uses: actions/checkout@v2 + + - name: golangci-lint + uses: golangci/golangci-lint-action@v2 + with: + version: v1.43.0 + args: --timeout 5m0s + + md-link-check: + name: Check markdown links + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v2 + + - name: Check if Markdown links are valid + uses: gaurav-nelson/github-action-markdown-link-check@v1 + with: + config-file: '.github/linters/mlc_config.json' + folder-path: "." + + sh-euox-pipefail-check: + name: Check for 'set -euox pipefail' in shell scripts + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v2 + + - name: Check that all shell scripts use 'set -euox pipefail' + run: bash .github/scripts/ensure_set_euox_pipefail.sh + + super-lint: + name: Run super linter + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v2 + + - name: Super-Linter + uses: github/super-linter/slim@v4 + env: + VALIDATE_MARKDOWN: true + VALIDATE_XML: true + VALIDATE_YAML: true + VALIDATE_OPENAPI: true + + VALIDATE_ALL_CODEBASE: true + LOG_LEVEL: WARN + + DEFAULT_BRANCH: main + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/markdown-ci/mlc_config.json b/.github/workflows/markdown-ci/mlc_config.json index d248115af..2a881e18a 100644 --- a/.github/workflows/markdown-ci/mlc_config.json +++ b/.github/workflows/markdown-ci/mlc_config.json @@ -1,3 +1,3 @@ { - "aliveStatusCodes": [200, 206, 999] -} \ No newline at end of file + "aliveStatusCodes": [0, 200, 206, 403, 999] +} diff --git a/.github/workflows/node-ci.yml b/.github/workflows/node-ci.yml index 185df740a..37735fca9 100644 --- a/.github/workflows/node-ci.yml +++ b/.github/workflows/node-ci.yml @@ -9,7 +9,7 @@ on: - '.gitbook/**' env: - NODE_CONFIGS_BASE: "/home/runner/work/cheqd-node/cheqd-node/tests/networks/docker_compose/node_configs" + NODE_CONFIGS_BASE: "/home/runner/work/cheqd-node/cheqd-node/tests/networks/docker-compose-localnet/node_configs" PACKAGE_NAME: "cheqd-node" VERSION: "1.0.0" # Package build scripts require version to be provided @@ -105,9 +105,9 @@ jobs: run: sudo chmod +x /home/runner/go/bin/cheqd-noded - name: Make intermediate archive - working-directory: ./build_tools + working-directory: ./build-tools run: | - ./build_tar.sh ${{ env.PACKAGE_NAME }} ${{ env.VERSION }} + ./build-tar.sh ${{ env.PACKAGE_NAME }} ${{ env.VERSION }} - name: Install fpm run: | @@ -115,15 +115,15 @@ jobs: sudo gem install --no-document fpm - name: Build deb # Depends on tar archive + working-directory: ./build-tools run: | - ./build_deb.sh ${{ env.PACKAGE_NAME }} ${{ env.VERSION }} - working-directory: ./build_tools + ./build-deb.sh ${{ env.PACKAGE_NAME }} ${{ env.VERSION }} - name: Store deb package artifact uses: actions/upload-artifact@v2 with: name: ${{ env.PACKAGE_NAME }}_${{ env.VERSION }}_amd64.deb - path: build_tools/output/${{ env.PACKAGE_NAME }}_${{ env.VERSION }}_amd64.deb + path: build-tools/output/${{ env.PACKAGE_NAME }}_${{ env.VERSION }}_amd64.deb build-node-image: name: Build cheqd-node image @@ -134,7 +134,10 @@ jobs: fetch-depth: 0 # FIXME: Starport requires full repository - name: Build - run: docker build -f docker/cheqd_node/Dockerfile --no-cache -t cheqd-node --build-arg UID=$(id -u) --build-arg GID=$(id -g) . # TODO: Get rid of UID and GID + run: | + export CHEQD_NODE_VERSION=local + docker compose --env-file docker/persistent-chains/docker-compose.env -f docker/persistent-chains/docker-compose.yml build --build-arg UID=$(id -u) --build-arg GID=$(id -g) + docker tag ghcr.io/cheqd/cheqd-node:local cheqd-node - name: Save run: docker save -o cheqd-node-image.tar cheqd-node @@ -162,7 +165,7 @@ jobs: run: docker load -i cheqd-node-image.tar - name: Build - run: docker build -f docker/single_image_testnet/Dockerfile --no-cache -t cheqd-testnet . + run: docker build -f tests/networks/docker-localnet/Dockerfile --no-cache -t cheqd-testnet . - name: Save run: docker save -o cheqd-testnet-image.tar cheqd-testnet @@ -198,23 +201,23 @@ jobs: uses: actions/checkout@v2 - name: Set up 4 validators + 2 observers node docker pool # TODO: Replace with testnet image - working-directory: ./tests/networks/docker_compose + working-directory: ./tests/networks/docker-compose-localnet run: | - ./gen_node_configs.sh - ./run_docker.sh + ./gen-node-configs.sh + ./run-docker.sh ./wait.sh '[[ $(curl -s -N localhost:26657/block | jq -cr '"'"'.result.block.last_commit.height'"'"') -gt 1 ]] && echo "Height is more than 1"' - name: Add observer node - working-directory: ./tests/networks/docker_compose + working-directory: ./tests/networks/docker-compose-localnet run: | - ./add_observer.sh + ./add-observer.sh cheqd-noded status -n tcp://localhost:26677 2>&1 ./wait.sh '[[ $(cheqd-noded status -n '"'"'tcp://localhost:26677'"'"' 2>&1 | wc -l) == 1 ]] && echo "New node returns status!"' - name: Promote observer to validator - working-directory: ./tests/networks/docker_compose + working-directory: ./tests/networks/docker-compose-localnet run: | - bash -x promote_validator.sh + bash -x promote-validator.sh cheqd-noded query staking validators --node "http://localhost:26657" | sed -nr 's/.*status: (.*?).*/\1/p' $1 | while read x; do [[ "BOND_STATUS_BONDED" == $x ]] && echo "Validator's status is bonded!" || exit 1 ; done ./wait.sh '[[ $(curl -s localhost:26657/block | sed -nr '"'"'s/.*signature": (.*?).*/\1/p'"'"' | wc -l) == 5 ]] && echo "There are 5 validators signatures in block!"' ./wait.sh '[[ $(curl -s localhost:26657/block | sed -nr '"'"'s/.*(signature": null).*/\1/p'"'"' | wc -l) == 0 ]] && echo "There are no null signatures in block!"' @@ -254,10 +257,10 @@ jobs: - uses: actions/checkout@v2 - name: Set up 4 validators + 2 observers node docker pool for local test run # Replace with testnet image - working-directory: ./tests/networks/docker_compose + working-directory: ./tests/networks/docker-compose-localnet run: | - ./gen_node_configs.sh - ./run_docker.sh + ./gen-node-configs.sh + ./run-docker.sh ./wait.sh '[[ $(curl -s -N localhost:26657/block | jq -cr '"'"'.result.block.last_commit.height'"'"') -gt 1 ]] && echo "Height is more than 1"' - name: Set up test environment @@ -292,7 +295,7 @@ jobs: - name: Run test net container run: | docker run -d --rm -p "26657:26657" cheqd-testnet - bash tests/networks/wait_for_chain.sh + bash tests/networks/tools/wait-for-chain.sh - name: Set up test environment working-directory: ./tests/e2e @@ -333,8 +336,8 @@ jobs: - name: Run test net container run: | - docker run -d --rm -p "26657:26657" -p "1317:1317" cheqd-testnet - bash tests/networks/wait_for_chain.sh + docker run -d --rm -p "26657:26657" -p "1317:1317" cheqd-testnet + bash tests/networks/tools/wait-for-chain.sh - name: Set up and run tests working-directory: tests/cli diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index ce8d7188e..19e81a9cb 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -20,10 +20,10 @@ jobs: - name: Set outputs id: cache run: | - # Exposes org_name/repository_name in lower case. Required by docker. + # Exposes org_name/repository_name in lower case. Required by Docker. echo "::set-output name=GITHUB_REPOSITORY_NAME::$(echo ${GITHUB_REPOSITORY,,})" - # Version is tag without `v` prefix. Reuired by deb package. + # Version is tag without `v` prefix. Required by Debian package. TAG=${GITHUB_REF/refs\/tags\//} echo "::set-output name=TAG::$TAG" @@ -39,7 +39,7 @@ jobs: with: go-version: 1.17 - - name: Get go protoc compiler plugins + - name: Get Go protoc compiler plugins env: GOLANG_PROTOBUF_VERSION: 1.3.5 GOGO_PROTOBUF_VERSION: 1.3.2 @@ -78,7 +78,7 @@ jobs: path: build/cheqd-noded build-deb-package: - name: Build deb package + name: Build Debian package runs-on: ubuntu-20.04 needs: [ build-binary, setup-workflow ] env: @@ -97,28 +97,28 @@ jobs: run: sudo chmod +x /home/runner/go/bin/cheqd-noded - name: Make intermediate archive - working-directory: ./build_tools + working-directory: ./build-tools run: | - ./build_tar.sh ${{ env.PACKAGE_NAME }} ${{ env.VERSION }} + ./build-tar.sh ${{ env.PACKAGE_NAME }} ${{ env.VERSION }} - name: Install fpm run: | sudo apt-get install ruby ruby-dev rubygems build-essential sudo gem install --no-document fpm - - name: Build deb # Depends on tar archive + - name: Build Debian package # Depends on tar archive + working-directory: ./build-tools run: | - ./build_deb.sh ${{ env.PACKAGE_NAME }} ${{ env.VERSION }} - working-directory: ./build_tools + ./build-deb.sh ${{ env.PACKAGE_NAME }} ${{ env.VERSION }} - - name: Store deb package artifact + - name: Store Debian package artifact uses: actions/upload-artifact@v2 with: name: ${{ env.PACKAGE_NAME }}_${{ env.VERSION }}_amd64.deb - path: build_tools/output/${{ env.PACKAGE_NAME }}_${{ env.VERSION }}_amd64.deb + path: build-tools/output/${{ env.PACKAGE_NAME }}_${{ env.VERSION }}_amd64.deb build-node-image: - name: Build cheqd-node image + name: Build cheqd-node Docker image runs-on: ubuntu-20.04 steps: - uses: actions/checkout@v2 @@ -126,7 +126,10 @@ jobs: fetch-depth: 0 # FIXME: Starport requires full repository - name: Build - run: docker build -f docker/cheqd_node/Dockerfile --no-cache -t cheqd-node --build-arg UID=$(id -u) --build-arg GID=$(id -g) . # TODO: Get rid of UID and GID + run: | + export CHEQD_NODE_VERSION=local + docker compose --env-file docker/persistent-chains/docker-compose.env -f docker/persistent-chains/docker-compose.yml build + docker tag ghcr.io/cheqd/cheqd-node:local cheqd-node - name: Save run: docker save -o cheqd-node-image.tar cheqd-node @@ -154,7 +157,7 @@ jobs: run: docker load -i cheqd-node-image.tar - name: Build - run: docker build -f docker/single_image_testnet/Dockerfile --no-cache -t cheqd-testnet . + run: docker build -f tests/networks/docker-localnet/Dockerfile --no-cache -t cheqd-testnet . - name: Save run: docker save -o cheqd-testnet-image.tar cheqd-testnet diff --git a/.gitignore b/.gitignore index e9ced95d6..7ee08aa16 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,9 @@ +### APP-SPECIFIC EXCLUSIONS ### +**/node_configs +.idea + +### GENERAL EXCLUSIONS ### + ### macOS ### # General .DS_Store @@ -30,14 +36,6 @@ Temporary Items .devcontainer .vscode -# Application specific -vue/node_modules -vue/src/store/generated -vue/dist -secret.yml -**/node_configs -.idea - # Python .pytest_cache **/__pycache__ diff --git a/SUMMARY.md b/SUMMARY.md index e254999d3..55e40ad25 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -21,9 +21,9 @@ * [Using cheqd Cosmos CLI for token transactions](docs/cheqd-cli/cheqd-cli-token-transactions.md) * [Building from source](docs/build-and-networks/README.md) * [Building and testing](docs/build-and-networks/README.md) - * [cheqd-testsnet docker image](docs/build-and-networks/local-docker-network.md) - * [Docker Compose Based Localnet](docs/build-and-networks/docker-compose.md) - * [Setting up a new network](docs/build-and-networks/build-and-networks.md) + * [Docker Based Localnet](docs/build-and-networks/local-docker-network.md) + * [Docker Compose Based Localnet](docs/build-and-networks/local-docker-compose-network.md) + * [Setting up a new network](docs/build-and-networks/manual-network-setup.md) * [Client-app Identity APIs](docs/identity-api/README.md) * [Error messages](docs/identity-api/identity-api-error-messages.md) diff --git a/build_tools/build_deb.sh b/build-tools/build-deb.sh similarity index 100% rename from build_tools/build_deb.sh rename to build-tools/build-deb.sh diff --git a/build_tools/build_tar.sh b/build-tools/build-tar.sh similarity index 100% rename from build_tools/build_tar.sh rename to build-tools/build-tar.sh diff --git a/build_tools/common.sh b/build-tools/common.sh similarity index 100% rename from build_tools/common.sh rename to build-tools/common.sh diff --git a/build_tools/postinst b/build-tools/postinst similarity index 99% rename from build_tools/postinst rename to build-tools/postinst index bb0c152a7..b4012aabe 100644 --- a/build_tools/postinst +++ b/build-tools/postinst @@ -1,4 +1,4 @@ -#!/bin/sh +#!/bin/bash CHEQD_USER_NAME=cheqd diff --git a/build_tools/postremove b/build-tools/postremove similarity index 100% rename from build_tools/postremove rename to build-tools/postremove diff --git a/docker/cheqd_node/Dockerfile b/docker/Dockerfile similarity index 52% rename from docker/cheqd_node/Dockerfile rename to docker/Dockerfile index ee99c500e..900de76c2 100644 --- a/docker/cheqd_node/Dockerfile +++ b/docker/Dockerfile @@ -1,4 +1,6 @@ -##### Build container ##### +############################################################### +### STAGE 1: Build node binary pre-requisites ### +############################################################### FROM golang:buster as builder @@ -6,8 +8,7 @@ RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \ && apt-get -y install --no-install-recommends \ curl \ libprotobuf-dev \ - wget - + && rm -rf /var/lib/apt/lists/* # Get go protoc compiler plugins. Taken from: tendermintdev/sdk-proto-gen:v0.2 ENV GOLANG_PROTOBUF_VERSION=1.3.5 \ @@ -23,15 +24,13 @@ RUN go get \ github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger@v${GRPC_GATEWAY_VERSION} \ github.com/regen-network/cosmos-proto/protoc-gen-gocosmos@latest - # Install buf RUN PREFIX="/usr/local" && \ VERSION="1.0.0-rc8" && \ curl -sSL "https://github.com/bufbuild/buf/releases/download/v${VERSION}/buf-$(uname -s)-$(uname -m).tar.gz" | \ tar -xvzf - -C "${PREFIX}" --strip-components 1 - -# App +# Copy pre-requisites before building the node binary WORKDIR /app COPY app ./app @@ -45,32 +44,49 @@ COPY Makefile . # Required to fetch version COPY .git . +# Make node binary RUN make proto-gen build +############################################################### +### STAGE 2: Build cheqd binary base container ### +############################################################### -##### Run container ##### +FROM ubuntu:focal AS base -FROM ubuntu:focal - -# Node binary +# Copy compiled node binary from Stage 1 COPY --from=builder /app/build/cheqd-noded /bin -# Runner script -COPY docker/cheqd_node/node-runner.sh /bin/node-runner -RUN chmod +x /bin/node-runner - -ARG CHEQD_HOME_DIR="/cheqd" - +# Set user directory and details +ARG CHEQD_HOME_DIR="/home/cheqd" ARG UID=1000 ARG GID=1000 -RUN groupadd --system --gid $GID cheqd && \ - useradd --system --create-home --home-dir ${CHEQD_HOME_DIR} --shell /bin/bash --gid cheqd --uid $UID cheqd +# Add cheqd user to use in the container +RUN groupadd --system --gid $GID cheqd \ + && useradd --system --create-home --home-dir ${CHEQD_HOME_DIR} --shell /bin/bash --gid cheqd --uid $UID cheqd WORKDIR ${CHEQD_HOME_DIR} USER cheqd -EXPOSE 26656 26657 +# Document default ports to expose to host +EXPOSE 26656 26657 26660 1317 9090 9091 + +# Define stop scenarios STOPSIGNAL SIGTERM +# Default entrypoint for cheqd-noded CLI usage ENTRYPOINT [ "cheqd-noded" ] + + +############################################################### +### STAGE 3: Build cheqd-node image ### +############################################################### + +FROM base AS node + +# Set runner script +COPY --chown=cheqd:cheqd docker/persistent-chains/entrypoint.sh /bin/node-start +RUN chmod +x /bin/node-start + +# Default entrypoint for cheqd-noded CLI usage +ENTRYPOINT [ "node-start" ] diff --git a/docker/persistent-chains/container.env b/docker/persistent-chains/container.env new file mode 100644 index 000000000..db5fd3921 --- /dev/null +++ b/docker/persistent-chains/container.env @@ -0,0 +1,55 @@ +# Environment variables used INSIDE the Docker container +# The INPUT REQUIRED section MUST be defined before running a node + +############################################################### +### INPUT REQUIRED: Node Initialisation Parameters ### +############################################################### + +# Externally-reachable address for the node +# Syntax is :26656 or :26656 +P2P_EXTERNAL_ADDRESS=":26656" + +# A custom human-readable name for this node +NODE_MONIKER="" + +############################################################### +### OPTIONAL: Per-node safe defaults ### +############################################################### +# Safe default settings isolated to individual nodes. +# These can be changed WITHOUT major network-wide consequences. + +# Default gas price. Higher gas prices can impact inclusion in mempool. +MIN_GAS_PRICES="25ncheq" + +# Comma-separated list of persistent peers to maintain connections to. +# Syntax is @: +P2P_PERSISTENT_PEERS="" + +# Max. send/receive rate throttle. Values below compute to ~20 MB/s. +# Can be increased if your node can handle higher network throughput. +P2P_SEND_RATE=20000000 +P2P_RECV_RATE=20000000 + +# Flip whether the node is a seed. For most users, the answer is false. +P2P_SEED_MODE=false + + +############################################################### +### CAUTION: Network-wide defaults ### +############################################################### +# Safe default settings used by most nodes across the network. +# Changing these CAN cause connectivity issues for your node. +# Only change if you understand the consequences. + + # Fetches correct genesis.json and seeds.txt from GitHub +NETWORK=testnet +# TODO: Replace with block-sync on TM upgrade +FASTSYNC_VERSION="v0" +# TODO: Check if empty block creation is actually turned off +CREATE_EMPTY_BLOCKS=false +# Max. message payload size. Default is ~10 KB +P2P_MAX_PACKET_MSG_PAYLOAD_SIZE=10240 +# Tendermint P2P port. Port exposed on host is more relevant to control access. +P2P_LADDR="tcp://0.0.0.0:26656" +# Tendermint RPC port. Port exposed on host is more relevant to control access. +RPC_LADDR="tcp://0.0.0.0:26657" diff --git a/docker/persistent-chains/docker-compose.env b/docker/persistent-chains/docker-compose.env new file mode 100644 index 000000000..bb7bab47f --- /dev/null +++ b/docker/persistent-chains/docker-compose.env @@ -0,0 +1,76 @@ +# Environment variables used by Docker Compose + +############################################################### +### REQUIRED: Node/network selection ### +############################################################### +# Version of cheqd-noded Docker image to pull from GitHub +# Needs to be changed in case of node version upgrades + +# Define which network you want to deploy a node +# Change value to "testnet" if targetting testnet +CHEQD_NETWORK="mainnet" + +# Define cheqd-noded software release version +# Current MAINNET recommended version: v0.3.1 +# Current TESTNET recommended version: v0.3.1 +CHEQD_NODE_VERSION="0.3.1" + + +############################################################### +### OPTIONAL: Host-side port mappings ### +############################################################### +# Ports to be exposed on the host machine/service +# Only required if you want to customise target->host mappings + +# Tendermint P2P +TM_P2P_PORT=26656 + +# Tendermint RPC API +TM_RPC_API_PORT=26657 + +# Prometheus metrics port (optional) +TM_PROMETHEUS_PORT=26660 + +# Cosmos REST API (optional) +COSMOS_REST_API_PORT=1317 + +# Cosmos gRPC API (optional) +COSMOS_GRPC_API_PORT=9090 + +# Cosmos gRPC-Gateway API (optional) +COSMOS_GRPC_GATEWAY_API_PORT=9191 + +############################################################### +### OPTIONAL: Docker volume configuration ### +############################################################### + +# Define a Docker volume source +# Default is set to the volume name referred in the docker-compose.yml file +CHEQD_DOCKER_VOLUME_SOURCE="cheqd_volume" + +# Define driver to be used for Docker volume +# Set to "local" for a single host, or others such as "overlay" for writing to +# an externally available volume through volume plugins, e.g. "nfs" +# You need to specify **driver_opts** in docker-compose.yml for your preferred driver +# Refer to: https://docs.docker.com/engine/extend/legacy_plugins/#/volume-plugins +CHEQD_DOCKER_VOLUME_DRIVER="local" + +# Set to true if created externally, outside of Docker Compose +CHEQD_DOCKER_VOLUME_IS_EXTERNAL="false" + +############################################################### +### OPTIONAL: Build arguments ### +############################################################### +# Environment variables only used during BUILD process + +# Home directory for cheqd user inside the container +CHEQD_HOME_DIR="/home/cheqd" + +# UID for cheqd user +UID=1000 + +# GID for cheqd user +GID=1000 + +# Build tag when rebuilding image +BUILD_TAG="local" diff --git a/docker/persistent-chains/docker-compose.yml b/docker/persistent-chains/docker-compose.yml new file mode 100644 index 000000000..a60bda95c --- /dev/null +++ b/docker/persistent-chains/docker-compose.yml @@ -0,0 +1,81 @@ +version: '3.7' + +services: + cheqd_node: + # CAUTION: Change CHEQD_NODE_VERSION to local in docker-compose.env if building your own image in section below + image: ghcr.io/cheqd/cheqd-node:${CHEQD_NODE_VERSION} + # OPTIONAL: Rebuild cheq-node Docker image, if you want build your own + # Default is to pull in the pre-published image on GitHub Container Registry + # SYNTAX: docker compose --env-file docker/persistent-chains/docker-compose.env -f docker/persistent-chains/docker-compose.yml build --no-cache + build: + context: ../.. + dockerfile: docker/Dockerfile + target: node + args: + CHEQD_HOME_DIR: ${CHEQD_HOME_DIR} + UID: ${UID} + GID: ${GID} + user: cheqd + configs: + - source: genesis + target: /genesis + - source: seeds + target: /seeds + ports: + # REQUIRED: Tendermint peer-to-peer (P2P) port + # Please do not change this unless you've also changed this in the external-address parameter in Tendermint config + - target: 26656 + published: ${TM_P2P_PORT} + protocol: tcp + mode: host + # RECOMMENDED: Tendermint RPC port + # Required if you want to use chqed-noded CLI directly from the host + - target: 26657 + published: ${TM_RPC_API_PORT} + protocol: tcp + mode: host + # OPTIONAL: Cosmos REST API port + # Only required if you plan on targetting REST API requests at your own node + - target: 1317 + published: ${COSMOS_REST_API_PORT} + protocol: tcp + mode: host + # RECOMMENDED: gRPC port + # Tendermint RPC requests (define above) are relayed to the gRPC port inside the container + # The Cosmos gRPC API has endpoints not otherwise available in Tendermint RPC and Cosmos REST API + - target: 9090 + published: ${COSMOS_GRPC_API_PORT} + protocol: tcp + mode: host + # OPTIONAL: gRPC-Gateway port + # gRPC-Gateway provides a RESTful interface to relay requests to gRPC API + - target: 9091 + published: ${COSMOS_GRPC_GATEWAY_API_PORT} + protocol: tcp + mode: host + # OPTIONAL: Prometheus metrics port + # Allows collection of Tendermint and Cosmos metrics using Prometheus + - target: 26660 + published: ${TM_PROMETHEUS_PORT} + protocol: tcp + mode: host + # Defines cheqd-noded configuration variables INSIDE the container + env_file: + - container.env + volumes: + - type: volume + source: ${CHEQD_DOCKER_VOLUME_SOURCE} # Defaults to cheqd_volume, defined below + target: /home/cheqd # Do NOT change mount point inside the container unless rebuilding the image + entrypoint: [ "node-start" ] + +configs: + genesis: + file: ../../persistent_chains/${CHEQD_NETWORK}/genesis.json + seeds: + file: ../../persistent_chains/${CHEQD_NETWORK}/seeds.txt + +volumes: + cheqd_volume: + name: cheqd_volume_${CHEQD_NETWORK} + driver: ${CHEQD_DOCKER_VOLUME_DRIVER} + external: ${CHEQD_DOCKER_VOLUME_IS_EXTERNAL} diff --git a/docker/cheqd_node/node-runner.sh b/docker/persistent-chains/entrypoint.sh similarity index 61% rename from docker/cheqd_node/node-runner.sh rename to docker/persistent-chains/entrypoint.sh index 4bb1f870d..31db039a5 100644 --- a/docker/cheqd_node/node-runner.sh +++ b/docker/persistent-chains/entrypoint.sh @@ -1,23 +1,42 @@ #!/bin/bash # Inits node configuration and runs the node. +# e -> exit immediately, u -> treat unset variables as errors and immediately, o -> sets the exit code to the rightmost command +set -euo pipefail -set -euox pipefail - +# within the container, $HOME=/home/cheqd CHEQD_ROOT_DIR="$HOME/.cheqdnode" # Init node config directory if [ ! -d "${CHEQD_ROOT_DIR}/config" ] then echo "Node config not found. Initializing." - cheqd-noded init $NODE_MONIKER --home + cheqd-noded init $NODE_MONIKER --home ${CHEQD_ROOT_DIR} else echo "Node config exists. Skipping initialization." fi -# Run configure -# `! -z` is used instead of `-n` to distinguilsh null and empty values +# Check if a genesis file has been passed in config +if [ -f "/genesis" ] +then + echo "Genesis file passed. Adding/replacing current genesis file." + cp /genesis ${CHEQD_ROOT_DIR}/config/genesis.json +else + echo "No genesis file passed. Skipping and retaining existing genesis." +fi + +# Check if a genesis file has been passed in config +if [ -f "/seeds" ] +then + echo "Seeds file passed. Replacing current seeds." + cp /seeds ${CHEQD_ROOT_DIR}/config/seeds.txt + cheqd-noded configure p2p seeds $(cat ${CHEQD_ROOT_DIR}/config/seeds.txt) +else + echo "No seeds file passed. Skipping and retaining existing seeds." +fi +# Run configure +# `! -z` is used instead of `-n` to distinguish null and empty values if [[ ! -z ${CREATE_EMPTY_BLOCKS+x} ]]; then cheqd-noded configure create-empty-blocks ${CREATE_EMPTY_BLOCKS}; fi if [[ ! -z ${FASTSYNC_VERSION+x} ]]; then cheqd-noded configure fastsync-version ${FASTSYNC_VERSION}; fi if [[ ! -z ${MIN_GAS_PRICES+x} ]]; then cheqd-noded configure min-gas-prices ${MIN_GAS_PRICES}; fi @@ -25,21 +44,11 @@ if [[ ! -z ${RPC_LADDR+x} ]]; then cheqd-noded configure rpc-laddr ${RPC_LADDR}; if [[ ! -z ${P2P_EXTERNAL_ADDRESS+x} ]]; then cheqd-noded configure p2p external-address ${P2P_EXTERNAL_ADDRESS}; fi if [[ ! -z ${P2P_LADDR+x} ]]; then cheqd-noded configure p2p laddr ${P2P_LADDR}; fi if [[ ! -z ${P2P_MAX_PACKET_MSG_PAYLOAD_SIZE+x} ]]; then cheqd-noded configure p2p max-packet-msg-payload-size ${P2P_MAX_PACKET_MSG_PAYLOAD_SIZE}; fi +if [[ ! -z ${P2P_SEEDS+x} ]]; then cheqd-noded configure p2p seeds ${P2P_SEEDS}; fi if [[ ! -z ${P2P_PERSISTENT_PEERS+x} ]]; then cheqd-noded configure p2p persistent-peers ${P2P_PERSISTENT_PEERS}; fi -if [[ ! -z ${P2P_RECV_RATE+x} ]]; then cheqd-noded configure p2p recv-rate ${P2P_RECV_RATE}; fi if [[ ! -z ${P2P_SEED_MODE+x} ]]; then cheqd-noded configure p2p seed-mode ${P2P_SEED_MODE}; fi -if [[ ! -z ${P2P_SEEDS+x} ]]; then cheqd-noded configure p2p seeds ${P2P_SEEDS}; fi +if [[ ! -z ${P2P_RECV_RATE+x} ]]; then cheqd-noded configure p2p recv-rate ${P2P_RECV_RATE}; fi if [[ ! -z ${P2P_SEND_RATE+x} ]]; then cheqd-noded configure p2p send-rate ${P2P_SEND_RATE}; fi - -# Update configs -set - # Disable ditailed command logging - -echo "Updating genesis" -echo "$GENESIS" | base64 --decode > $CHEQD_ROOT_DIR/config/genesis.json - -set -x # Re-enable ditailed command logging - # Run node -NODE_ARGS=${NODE_ARGS:-} # Allo node args to be empty -cheqd-noded start $NODE_ARGS +cheqd-noded start diff --git a/docs/build-and-networks/README.md b/docs/build-and-networks/README.md index d1dd5cb69..da2be5c9c 100644 --- a/docs/build-and-networks/README.md +++ b/docs/build-and-networks/README.md @@ -1,8 +1,8 @@ -# Building and testing +# Building `cheqd-noded` -## Building `cheqd-node` from source +## Building from source -### Prerequisites +Prerequisites: * Install [Go](https://golang.org/doc/install) * Install [Starport](https://docs.starport.network/guide/install.html) @@ -19,9 +19,11 @@ To look up binary's location run: which cheqd-noded ``` -## Building node in docker +## Building in docker + +Use this [instruction](../setup-and-configure/docker-install.md). -Use this [instruction](build-and-networks.md). +# Running a network ## Running local network using starport @@ -40,6 +42,10 @@ starport serve Your blockchain in development can be configured with `config.yml`. To learn more see the [reference](https://github.com/tendermint/starport#documentation). +## Running local network in single docker image + +Use the [Docker localnet instructions](local-docker-network.md). + ## Running local network using docker compose -Use the [Docker Compose instructions](docker-compose.md). +Use the [Docker Compose localnet instructions](local-docker-compose-network.md). diff --git a/docs/build-and-networks/docker-compose.md b/docs/build-and-networks/local-docker-compose-network.md similarity index 94% rename from docs/build-and-networks/docker-compose.md rename to docs/build-and-networks/local-docker-compose-network.md index 7fed27eaa..cbd503289 100644 --- a/docs/build-and-networks/docker-compose.md +++ b/docs/build-and-networks/local-docker-compose-network.md @@ -13,7 +13,7 @@ The set of scripts to generate configurations for a network of four nodes and ru 1. Build docker image: - See [the instruction](local-docker-network.md). + See [the instruction](../setup-and-configure/docker-install.md). 2. Build cheqd-node: diff --git a/docs/build-and-networks/local-docker-network.md b/docs/build-and-networks/local-docker-network.md index bb7cac1a5..e44ab9f09 100644 --- a/docs/build-and-networks/local-docker-network.md +++ b/docs/build-and-networks/local-docker-network.md @@ -18,14 +18,14 @@ docker pull ghcr.io/cheqd/cheqd-testnet:latest ## Prerequisites -* Build `cheqd-node` image first. See the [instruction](build-and-networks.md). +* Build `cheqd-node` image first. See the [instruction](../setup-and-configure/docker-install.md). ## Building To build the image: * Go to the repository root -* Run `docker build -f docker/single_image_testnet/Dockerfile -t cheqd-testnet .` +* Run `docker build -f tests/networks/docker-localnet/Dockerfile -t cheqd-testnet .` ## Running diff --git a/docs/build-and-networks/build-and-networks.md b/docs/build-and-networks/manual-network-setup.md similarity index 99% rename from docs/build-and-networks/build-and-networks.md rename to docs/build-and-networks/manual-network-setup.md index cc386c3cd..551796ecd 100644 --- a/docs/build-and-networks/build-and-networks.md +++ b/docs/build-and-networks/manual-network-setup.md @@ -1,4 +1,4 @@ -# setting-up-a-new-network +# Setting up a new network manually ## Creating a new network from genesis diff --git a/docs/setup-and-configure/README.md b/docs/setup-and-configure/README.md index 5a99fc002..bdce5fc89 100644 --- a/docs/setup-and-configure/README.md +++ b/docs/setup-and-configure/README.md @@ -4,7 +4,7 @@ This document describes how to use install and configure a new instance of `cheqd-node` from pre-built packages and adding it to an existing network \(such as the cheqd testnet\) as an observer or validator. -For other scenarios, please see [setting up a new network from scratch](../build-and-networks/build-and-networks.md) and [building `cheqd-node` from source](../build-and-networks/README.md). +For other scenarios, please see [setting up a new network from scratch](../build-and-networks/manual-network-setup.md) and [building `cheqd-node` from source](../build-and-networks/README.md). ## Pre-requisites diff --git a/docs/setup-and-configure/binary-install.md b/docs/setup-and-configure/binary-install.md index 2a0acfdc4..5fd54f9b1 100644 --- a/docs/setup-and-configure/binary-install.md +++ b/docs/setup-and-configure/binary-install.md @@ -21,7 +21,7 @@ The [node setup guide provides pre-requisites](README.md) needed before the step It is highly recommended to run the `cheqd-node` as a system service using a supervisor such as `systemd`. - Our Debian package uses [postinst](https://github.com/cheqd/cheqd-node/blob/main/build_tools/postinst) script for setting up our binary as a service. The same tool can be used to set up the binary as a service. + Our Debian package uses [postinst](../../build-tools/postinst) script for setting up our binary as a service. The same tool can be used to set up the binary as a service. There is only one input parameter for `postinst` script, it's a path to where binary is. diff --git a/docs/setup-and-configure/docker-install.md b/docs/setup-and-configure/docker-install.md index 2e4ee777c..bb92af832 100644 --- a/docs/setup-and-configure/docker-install.md +++ b/docs/setup-and-configure/docker-install.md @@ -21,7 +21,7 @@ docker pull ghcr.io/cheqd/cheqd-node:latest To build the image: * Go to the repository root; -* Run `docker build -f docker/cheqd_node/Dockerfile -t cheqd-node .`. +* Run `docker build -f docker/Dockerfile -t cheqd-node .`. Default home directory for `cheqd` user is `/cheqd`. It can be overridden via `CHEQD_HOME_DIR` build argument. Example: `--build-arg CHEQD_HOME_DIR=/home/cheqd`. diff --git a/tests/networks/docker_compose/add_observer.sh b/tests/networks/docker-compose-localnet/add-observer.sh similarity index 93% rename from tests/networks/docker_compose/add_observer.sh rename to tests/networks/docker-compose-localnet/add-observer.sh index a6e832388..d365137f7 100755 --- a/tests/networks/docker_compose/add_observer.sh +++ b/tests/networks/docker-compose-localnet/add-observer.sh @@ -3,7 +3,7 @@ set -euox pipefail sudo chown -R cheqd:cheqd "/home/runner/cheqd/" -NODE_CONFIGS_BASE="/home/runner/work/cheqd-node/cheqd-node/tests/networks/docker_compose/node_configs" +NODE_CONFIGS_BASE="/home/runner/work/cheqd-node/cheqd-node/tests/networks/docker-compose-localnet/node_configs" sudo -u cheqd cheqd-noded init node5 NODE0_ID=$(cat "${NODE_CONFIGS_BASE}/node0/node_id.txt") diff --git a/tests/networks/docker_compose/docker-compose.yml b/tests/networks/docker-compose-localnet/docker-compose.yml similarity index 75% rename from tests/networks/docker_compose/docker-compose.yml rename to tests/networks/docker-compose-localnet/docker-compose.yml index a4d1a7ef1..409e39f2f 100644 --- a/tests/networks/docker_compose/docker-compose.yml +++ b/tests/networks/docker-compose-localnet/docker-compose.yml @@ -7,7 +7,8 @@ services: - "26656:26656" # p2p - "26657:26657" # rpc volumes: - - ./node_configs/node0:/cheqd + - ./node_configs/node0:/home/cheqd + entrypoint: cheqd-noded command: start --rpc.laddr tcp://0.0.0.0:26657 node1: @@ -16,7 +17,8 @@ services: - "26659:26656" # p2p - "26660:26657" # rpc volumes: - - ./node_configs/node1:/cheqd + - ./node_configs/node1:/home/cheqd + entrypoint: cheqd-noded command: start --rpc.laddr tcp://0.0.0.0:26657 --p2p.persistent_peers "${NODE_0_ID}@node0:26656" node2: @@ -25,7 +27,8 @@ services: - "26662:26656" # p2p - "26663:26657" # rpc volumes: - - ./node_configs/node2:/cheqd + - ./node_configs/node2:/home/cheqd + entrypoint: cheqd-noded command: start --rpc.laddr tcp://0.0.0.0:26657 --p2p.persistent_peers "${NODE_0_ID}@node0:26656" node3: @@ -34,7 +37,8 @@ services: - "26665:26656" # p2p - "26666:26657" # rpc volumes: - - ./node_configs/node3:/cheqd + - ./node_configs/node3:/home/cheqd + entrypoint: cheqd-noded command: start --rpc.laddr tcp://0.0.0.0:26657 --p2p.persistent_peers "${NODE_0_ID}@node0:26656" observer0: @@ -43,7 +47,8 @@ services: - "26668:26656" # p2p - "26669:26657" # rpc volumes: - - ./node_configs/observer0:/cheqd + - ./node_configs/observer0:/home/cheqd + entrypoint: cheqd-noded command: start --rpc.laddr tcp://0.0.0.0:26657 --minimum-gas-prices "1cheq" --p2p.persistent_peers "${NODE_0_ID}@node0:26656" observer1: @@ -52,5 +57,6 @@ services: - "26671:26656" # p2p - "26672:26657" # rpc volumes: - - ./node_configs/observer1:/cheqd + - ./node_configs/observer1:/home/cheqd + entrypoint: cheqd-noded command: start --rpc.laddr tcp://0.0.0.0:26657 --minimum-gas-prices "1cheq" --p2p.persistent_peers "${NODE_0_ID}@node0:26656" diff --git a/tests/networks/docker_compose/gen_node_configs.sh b/tests/networks/docker-compose-localnet/gen-node-configs.sh similarity index 98% rename from tests/networks/docker_compose/gen_node_configs.sh rename to tests/networks/docker-compose-localnet/gen-node-configs.sh index 872161f91..d0f2ce00f 100755 --- a/tests/networks/docker_compose/gen_node_configs.sh +++ b/tests/networks/docker-compose-localnet/gen-node-configs.sh @@ -17,7 +17,8 @@ fi cheqd_noded_docker() { docker run --rm \ - -v "$(pwd)":"/cheqd" \ + -v "$(pwd)":"/home/cheqd" \ + --entrypoint "cheqd-noded" \ cheqd-node "$@" } diff --git a/tests/networks/docker_compose/promote_validator.sh b/tests/networks/docker-compose-localnet/promote-validator.sh similarity index 100% rename from tests/networks/docker_compose/promote_validator.sh rename to tests/networks/docker-compose-localnet/promote-validator.sh diff --git a/tests/networks/docker_compose/run_docker.sh b/tests/networks/docker-compose-localnet/run-docker.sh similarity index 88% rename from tests/networks/docker_compose/run_docker.sh rename to tests/networks/docker-compose-localnet/run-docker.sh index 9434c2b49..00be20b3d 100755 --- a/tests/networks/docker_compose/run_docker.sh +++ b/tests/networks/docker-compose-localnet/run-docker.sh @@ -8,7 +8,8 @@ set -euox pipefail cheqd_noded_docker() { docker run --rm \ - -v "$(pwd)":"/cheqd" \ + -v "$(pwd)":"/home/cheqd" \ + --entrypoint "cheqd-noded" \ cheqd-node "$@" } diff --git a/tests/networks/docker_compose/wait.sh b/tests/networks/docker-compose-localnet/wait.sh similarity index 100% rename from tests/networks/docker_compose/wait.sh rename to tests/networks/docker-compose-localnet/wait.sh diff --git a/docker/single_image_testnet/Dockerfile b/tests/networks/docker-localnet/Dockerfile similarity index 74% rename from docker/single_image_testnet/Dockerfile rename to tests/networks/docker-localnet/Dockerfile index 8f490f1ef..da5975c4e 100644 --- a/docker/single_image_testnet/Dockerfile +++ b/tests/networks/docker-localnet/Dockerfile @@ -9,17 +9,18 @@ RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \ USER cheqd -COPY docker/single_image_testnet/gen_node_configs.sh . -COPY docker/single_image_testnet/supervisord.conf /etc +COPY --chown=cheqd:cheqd tests/networks/docker-localnet/gen-node-configs.sh ./ +COPY tests/networks/docker-localnet/supervisord.conf /etc -EXPOSE 26657 26659 - -RUN ./gen_node_configs.sh +RUN chmod +x ./gen-node-configs.sh +RUN ./gen-node-configs.sh # Update persistent peers list for at least one node RUN NODE_0_ID=$(cheqd-noded tendermint show-node-id); \ PERSISTENT_PEERS="$NODE_0_ID@127.0.0.1:26656"; \ sed -i $extension "s/persistent_peers = \"\"/persistent_peers = \"$PERSISTENT_PEERS\"/g" .cheqdnode/config/config.toml +EXPOSE 26657 26659 + ENTRYPOINT ["/usr/bin/supervisord", "-c", "/etc/supervisord.conf"] STOPSIGNAL SIGTERM diff --git a/docker/single_image_testnet/gen_node_configs_new.sh b/tests/networks/docker-localnet/gen-node-configs-new.sh old mode 100755 new mode 100644 similarity index 100% rename from docker/single_image_testnet/gen_node_configs_new.sh rename to tests/networks/docker-localnet/gen-node-configs-new.sh diff --git a/docker/single_image_testnet/gen_node_configs.sh b/tests/networks/docker-localnet/gen-node-configs.sh old mode 100755 new mode 100644 similarity index 100% rename from docker/single_image_testnet/gen_node_configs.sh rename to tests/networks/docker-localnet/gen-node-configs.sh diff --git a/docker/single_image_testnet/supervisord.conf b/tests/networks/docker-localnet/supervisord.conf similarity index 96% rename from docker/single_image_testnet/supervisord.conf rename to tests/networks/docker-localnet/supervisord.conf index b8af0509c..5bae819d2 100644 --- a/docker/single_image_testnet/supervisord.conf +++ b/tests/networks/docker-localnet/supervisord.conf @@ -18,7 +18,7 @@ strip_ansi = false [program:node0] command=cheqd-noded start --p2p.laddr tcp://127.0.0.1:26656 --grpc.address 0.0.0.0:9090 --rpc.laddr tcp://0.0.0.0:26657 -directory=/cheqd +directory=/home/cheqd stdout_logfile=/tmp/node0.log stderr_logfile=/tmp/node0.log diff --git a/tests/networks/docker_compose.md b/tests/networks/docker_compose.md deleted file mode 100644 index 5ae60b7b3..000000000 --- a/tests/networks/docker_compose.md +++ /dev/null @@ -1,67 +0,0 @@ -# Docker Compose Based Localnet - -## Description - -The set of scripts to generate configurations for a network of four nodes and run it locally in docker-compose. - -## Prerequisites - -* [Starport](https://docs.starport.network/guide/install.html) -* docker-compose - -## How to run - -1. Build docker image: - - See [the instruction](../../docs/build-and-networks/build-and-networks.md). - -2. Build cheqd-noded: - - ```text - starport chain build - ``` - -3. Generate node configurations: - - Run: `gen_node_configs.sh`. - -4. Run docker-compose: - - Run: `run_docker.sh`. - -## Result - -### Nodes - -This will setup 4 nodes listening on the following ports: - -* Node0: - * p2p: 26656 - * rpc: 26657 -* Node1: - * p2p: 26659 - * rpc: 26660 -* Node2: - * p2p: 26662 - * rpc: 26663 -* Node3: - * p2p: 26665 - * rpc: 26666 - -You can tests connection to a node using browser: `http://localhost:`. Example for the first node: `http://localhost:26657`. - -### Accounts - -Also, there will be 4 keys generated and corresponding genesis accounts created for node operators: - -* operator0; -* operator1; -* operator2; -* operator3; - -When connecting using CLI, point path to home directory: `--home node_configs/client`. - -## CLI commands: - -See [the reference](../../docs/cheqd-cli/README.md) to learn about the most common CLI flows. - diff --git a/tests/networks/local_node.md b/tests/networks/local_node.md deleted file mode 100644 index edbda2687..000000000 --- a/tests/networks/local_node.md +++ /dev/null @@ -1,48 +0,0 @@ -# Single Node Localnet - -## Description - -The script to generate configuration for a network of one node and run it locally without docker. - -> Warning: this script removes all files in `$HOME/.cheqdnode` directory. Backup it first. - -## Prerequisites - -* [Starport](https://docs.starport.network/guide/install.html) - -## How to run - -1. Build cheqd-noded: - - ```text - starport chain build - ``` - -2. Generate network configuration: - - Run: `gen_node_config.sh`. - -4. Run single node network: - - Run: `cheqd-noded start`. - -## Result - -### Nodes - -This will setup 1 node listening on the following ports: - -* p2p: 26656 -* rpc: 26657 - -You can tests connection to the node using browser: `http://localhost:`. Example for the first node: `http://localhost:26657`. - -### Accounts - -Also, there will be 1 key generated and corresponding genesis accounts created for node operator: - -* node_operator; - -## CLI commands: - -See [the reference](https://github.com/cheqd/cheqd-node/blob/main/docs/cheqd-cli/README.md) to learn about the most common CLI flows. diff --git a/tests/networks/local_node/gen_node_config.sh b/tests/networks/local_node/gen_node_config.sh deleted file mode 100755 index 1d3a3fd20..000000000 --- a/tests/networks/local_node/gen_node_config.sh +++ /dev/null @@ -1,80 +0,0 @@ -#!/bin/bash - -set -euox pipefail - -# sed in macos requires extra argument - -sed_extension='' -if [[ "$OSTYPE" == "linux-gnu"* ]]; then - sed_extension='' -elif [[ "$OSTYPE" == "darwin"* ]]; then - sed_extension='.orig' -fi - - -CHAIN_ID="cheqd" - -rm -rf "$HOME/.cheqdnode" - -cheqd-noded init "local_node" --chain-id $CHAIN_ID -sed -i $sed_extension 's/"stake"/"ncheq"/' $HOME/.cheqdnode/config/genesis.json -sed -i $sed_extension 's/minimum-gas-prices = ""/minimum-gas-prices = "25ncheq"/g' $HOME/.cheqdnode/config/app.toml - - -cheqd-noded keys add "node_operator" --keyring-backend "test" -cheqd-noded add-genesis-account "node_operator" 20000000000000000ncheq --keyring-backend "test" - -echo "##### [Node 0] Switch on REST API" - -sed -i $sed_extension 's/enable = false/enable = true/g' "$HOME/.cheqdnode/config/app.toml" - -echo "##### [Node 0] Adjust consensus timeouts" - -sed -i $sed_extension 's/timeout_propose = "3s"/timeout_propose = "500ms"/g' "$HOME/.cheqdnode/config/config.toml" -sed -i $sed_extension 's/timeout_prevote = "1s"/timeout_prevote = "500ms"/g' "$HOME/.cheqdnode/config/config.toml" -sed -i $sed_extension 's/timeout_precommit = "1s"/timeout_precommit = "500ms"/g' "$HOME/.cheqdnode/config/config.toml" -sed -i $sed_extension 's/timeout_commit = "5s"/timeout_commit = "500ms"/g' "$HOME/.cheqdnode/config/config.toml" - -echo "##### Adding test accounts to the genesis" - -GENESIS="$HOME/.cheqdnode/config/genesis.json" - -BASE_ACCOUNT_1="cheqd1rnr5jrt4exl0samwj0yegv99jeskl0hsxmcz96" -# Mnemonic: sketch mountain erode window enact net enrich smoke claim kangaroo another visual write meat latin bacon pulp similar forum guilt father state erase bright -cat <<< "$(jq '.app_state.bank.balances += [{"address": "'${BASE_ACCOUNT_1}'", "coins": [{"denom": "ncheq", "amount": "100001000000000000"}] }]' "$GENESIS")" > "$GENESIS" -cat <<< "$(jq '.app_state.auth.accounts += [{"@type": "/cosmos.auth.v1beta1.BaseAccount","address": "'${BASE_ACCOUNT_1}'", "pub_key": null,"account_number": "0","sequence": "0"}]' "$GENESIS")" > "$GENESIS" - -BASE_ACCOUNT_2="cheqd1l9sq0se0jd3vklyrrtjchx4ua47awug5vsyeeh" -# Mnemonic: ugly dirt sorry girl prepare argue door man that manual glow scout bomb pigeon matter library transfer flower clown cat miss pluck drama dizzy -cat <<< "$(jq '.app_state.bank.balances += [{"address": "'${BASE_ACCOUNT_2}'", "coins": [{"denom": "ncheq", "amount": "100001000000000000"}] }]' "$GENESIS")" > $GENESIS -cat <<< "$(jq '.app_state.auth.accounts += [{"@type": "/cosmos.auth.v1beta1.BaseAccount","address": "'${BASE_ACCOUNT_2}'", "pub_key": null,"account_number": "0","sequence": "0"}]' "$GENESIS")" > "$GENESIS" - -BASE_VESTING_ACCOUNT="cheqd1lkqddnapqvz2hujx2trpj7xj6c9hmuq7uhl0md" -# Mnemonic: coach index fence broken very cricket someone casino dial truth fitness stay habit such three jump exotic spawn planet fragile walk enact angry great -BASE_VESTING_COIN="{\"denom\":\"ncheq\",\"amount\":\"10001000000000000\"}" -cat <<< "$(jq '.app_state.bank.balances += [{"address": "'${BASE_VESTING_ACCOUNT}'", "coins": [{"denom": "ncheq", "amount": "5000000000000000"}] }]' "$GENESIS")" > "$GENESIS" -cat <<< "$(jq '.app_state.auth.accounts += [{"@type": "/cosmos.vesting.v1beta1.BaseVestingAccount", "base_account": {"address": "'${BASE_VESTING_ACCOUNT}'","pub_key": null,"account_number": "0","sequence": "0"}, "original_vesting": ['${BASE_VESTING_COIN}'], "delegated_free": [], "delegated_vesting": [], "end_time": "1630362459"}]' "$GENESIS")" > "$GENESIS" - -CONTINOUS_VESTING_ACCOUNT="cheqd1353p46macvn444rupg2jstmx3tmz657yt9gl4l" -# Mnemonic: phone worry flame safe panther dirt picture pepper purchase tiny search theme issue genre orange merit stove spoil surface color garment mind chuckle image -cat <<< "$(jq '.app_state.bank.balances += [{"address": "'${CONTINOUS_VESTING_ACCOUNT}'", "coins": [{"denom": "ncheq", "amount": "5000000000000000"}] }]' "$GENESIS")" > "$GENESIS" -cat <<< "$(jq '.app_state.auth.accounts += [{"@type": "/cosmos.vesting.v1beta1.ContinuousVestingAccount", "base_vesting_account": { "base_account": {"address": "'${CONTINOUS_VESTING_ACCOUNT}'","pub_key": null,"account_number": "0","sequence": "0"}, "original_vesting": ['${BASE_VESTING_COIN}'], "delegated_free": [], "delegated_vesting": [], "end_time": "1630362459"}, "start_time": "1630352459"}]' "$GENESIS")" > "$GENESIS" - -DELAYED_VESTING_ACCOUNT="cheqd1njwu33lek5jt4kzlmljkp366ny4qpqusahpyrj" -# Mnemonic: pilot text keen deal economy donkey use artist divide foster walk pink breeze proud dish brown icon shaft infant level labor lift will tomorrow -cat <<< "$(jq '.app_state.bank.balances += [{"address": "'${DELAYED_VESTING_ACCOUNT}'", "coins": [{"denom": "ncheq", "amount": "5000000000000000"}] }]' "$GENESIS")" > "$GENESIS" -cat <<< "$(jq '.app_state.auth.accounts += [{"@type": "/cosmos.vesting.v1beta1.DelayedVestingAccount", "base_vesting_account": { "base_account": {"address": "'${DELAYED_VESTING_ACCOUNT}'","pub_key": null,"account_number": "0","sequence": "0"}, "original_vesting": ['${BASE_VESTING_COIN}'], "delegated_free": [], "delegated_vesting": [], "end_time": "1630362459"}}]' "$GENESIS")" > "$GENESIS" - -PERIODIC_VESTING_ACCOUNT="cheqd1uyngr0l3xtyj07js9sdew9mk50tqeq8lghhcfr" -# Mnemonic: want merge flame plate trouble moral submit wing whale sick meat lonely yellow lens enable oyster slight health vast weird radar mesh grab olive -cat <<< "$(jq '.app_state.bank.balances += [{"address": "'${PERIODIC_VESTING_ACCOUNT}'", "coins": [{"denom": "ncheq", "amount": "5000000000000000"}] }]' "$GENESIS")" > "$GENESIS" -cat <<< "$(jq '.app_state.auth.accounts += [{"@type": "/cosmos.vesting.v1beta1.PeriodicVestingAccount", "base_vesting_account": { "base_account": {"address": "'${PERIODIC_VESTING_ACCOUNT}'","pub_key": null,"account_number": "0","sequence": "0"}, "original_vesting": ['${BASE_VESTING_COIN}'], "delegated_free": [], "delegated_vesting": [], "end_time": "1630362459"}, "start_time": "1630362439", "vesting_periods": [{"length": "20", "amount": ['${BASE_VESTING_COIN}']}]}]' "$GENESIS")" > "$GENESIS" - - -NODE_ID=$(cheqd-noded tendermint show-node-id) -NODE_VAL_PUBKEY=$(cheqd-noded tendermint show-validator) - -cheqd-noded gentx "node_operator" 1000000000000000ncheq --chain-id $CHAIN_ID --node-id $NODE_ID --pubkey $NODE_VAL_PUBKEY --keyring-backend "test" - -cheqd-noded collect-gentxs -cheqd-noded validate-genesis diff --git a/tests/networks/wait_for_chain.sh b/tests/networks/tools/wait-for-chain.sh similarity index 100% rename from tests/networks/wait_for_chain.sh rename to tests/networks/tools/wait-for-chain.sh