Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unified network configuration generation #267

Merged
merged 97 commits into from
Feb 11, 2022
Merged
Show file tree
Hide file tree
Changes from 86 commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
0ce6d9e
Refactor: Docker Compose Easy Setup
jay-dee7 Dec 10, 2021
e85cba2
Changed action to reflect the updates
Eengineer1 Dec 10, 2021
f52d351
Add: long-syntax for ports in docker-compose config
jay-dee7 Dec 13, 2021
d20aab7
File structure and workflows
Eengineer1 Dec 13, 2021
95c8c9e
Merge branch 'new-docker-compose' of https://github.com/cheqd/cheqd-n…
Eengineer1 Dec 13, 2021
59fe235
Switched permissions
Eengineer1 Dec 13, 2021
3978c17
Copy as root then own it
Eengineer1 Dec 13, 2021
51e96b6
Added permissions
Eengineer1 Dec 13, 2021
52fca4d
Created localnet setup extracted from workflow
Eengineer1 Dec 13, 2021
687c8cc
Removed volume from Dockerfile
Eengineer1 Dec 14, 2021
03ce3ef
Switched directory to match changes
Eengineer1 Dec 14, 2021
137ab28
Fixed to match new image settings
Eengineer1 Dec 14, 2021
846c683
External laddr availability
Eengineer1 Dec 14, 2021
d2a248d
Switched test port to avoid clashes
Eengineer1 Dec 14, 2021
febc36b
Bypass defaults
Eengineer1 Dec 14, 2021
926bbdb
Removed excess
Eengineer1 Dec 14, 2021
7ab6769
Removed moniker name in container env vars
jay-dee7 Dec 15, 2021
b7f3e3c
Fixed supervisor config to match dirs
Eengineer1 Dec 15, 2021
0928943
Removed unwanted quotes
Eengineer1 Dec 15, 2021
fa50c10
Merge branch 'main' into new-docker-compose
Eengineer1 Dec 24, 2021
e099c33
Merge branch 'main' into new-docker-compose
Eengineer1 Dec 24, 2021
fb60a5c
Switched to reflect main
Eengineer1 Dec 24, 2021
3ad8ad0
Single Dockerfile for mainnet & testnet
Eengineer1 Dec 24, 2021
5abb084
Adjusted to old dir
Eengineer1 Dec 24, 2021
fdf6e02
Removed excess
Eengineer1 Dec 24, 2021
f981be3
Upgraded testnet version beforehand
Eengineer1 Dec 24, 2021
63303b3
Added whole home dir volume
Eengineer1 Dec 24, 2021
9154844
Multiple driver options & fixes
Eengineer1 Jan 10, 2022
bb91640
Testnet ref typo fix
Eengineer1 Jan 10, 2022
47506cf
Consistent across mainnet & testnet
Eengineer1 Jan 10, 2022
eaf75bf
Moved environment variables to host
Eengineer1 Jan 10, 2022
8089aec
Merge branch 'main' into new-docker-compose
askolesov Jan 11, 2022
c471a4d
Move single image localnet to tests
askolesov Jan 11, 2022
243901b
Fix naming of build tools
askolesov Jan 11, 2022
75886df
Fx liinks in workflows
askolesov Jan 11, 2022
7be8d5b
Move docker localnet to tests
askolesov Jan 11, 2022
6212660
Fix documentation
askolesov Jan 11, 2022
4b7dc28
Fix case
askolesov Jan 11, 2022
763aeae
Fix broken links
askolesov Jan 11, 2022
582e6a4
Allow 0, 403 status codes for link checker
askolesov Jan 11, 2022
afc76b0
Fix broken links
askolesov Jan 11, 2022
30589c5
Fix file names
askolesov Jan 11, 2022
c5fc54a
Fx links, dockerfile
askolesov Jan 11, 2022
28c6946
Fix test pipeline
askolesov Jan 11, 2022
29969bf
Fix test pipeline
askolesov Jan 11, 2022
68ac40a
Fix test
askolesov Jan 11, 2022
5e9cd5b
Revert volume names
askolesov Jan 11, 2022
44770e7
Fix corner case where host ip is invalid at first launch
askolesov Jan 11, 2022
71aa4c9
Optimize docker file
askolesov Jan 11, 2022
042ed46
Improve gen-network-config
askolesov Jan 12, 2022
f6c1f7e
Update git ignore
askolesov Jan 12, 2022
69ce3bf
Update cheqd-localnet image to run several nodes inside again
askolesov Jan 12, 2022
89f1429
Corrected spelling in filename
ankurdotb Jan 12, 2022
d71edb4
Make docker-compose localnet to use unified gen-net-conf base script
askolesov Jan 12, 2022
1d82709
Fix script name in the lint pipeline
askolesov Jan 12, 2022
72f4e97
Fix permissions in the test pipeline
askolesov Jan 12, 2022
dcd9ded
Replace waiter script with the new one
askolesov Jan 12, 2022
dd900e4
Typo fixes in GitHub Actions release workflow
ankurdotb Jan 12, 2022
33be740
Cosmetic switch to put exposed ports in same order as in config files
ankurdotb Jan 12, 2022
777d33b
Swapping parameter order around in node-runner.sh
ankurdotb Jan 12, 2022
ea70f3b
Added explanatory comments to Docker Compose file
ankurdotb Jan 12, 2022
4f46eb1
Added extended descriptions for inside-container configuration file f…
ankurdotb Jan 12, 2022
a1f875d
Update Dockerfile
ankurdotb Jan 13, 2022
78c1b45
Modified configuration for volumes in Docker Compose file
ankurdotb Jan 13, 2022
ed23e5b
Extended descriptions in host environment variables
ankurdotb Jan 13, 2022
3c28fa6
Merge remote-tracking branch 'origin/new-docker-compose' into unified…
askolesov Jan 13, 2022
47fc9c4
Update .Dockerignore
ankurdotb Jan 13, 2022
66de48b
Update release.yml
ankurdotb Jan 13, 2022
804983f
Update .gitignore
ankurdotb Jan 13, 2022
8c28b3a
Fix paths
askolesov Jan 13, 2022
22a61a0
More path fixes
askolesov Jan 13, 2022
4fe2e95
Fix paths
askolesov Jan 13, 2022
9155127
Fix typos
askolesov Jan 13, 2022
62b53d5
Fix failing tests
askolesov Jan 13, 2022
ff97c4f
Changed node-runner script to copy genesis and seed config data from …
ankurdotb Jan 14, 2022
50c60ad
Optimise Dockerfile by removing unnecessary requirements
ankurdotb Jan 14, 2022
e1104a6
Fixed errors due to incorrect commenting style in Compose environment…
ankurdotb Jan 14, 2022
18fae6e
Fixed incorrect variable names in Docker Compose file
ankurdotb Jan 14, 2022
52d7c3c
Merge branch 'new-docker-compose' into unified-net-config-generation
askolesov Jan 15, 2022
1ce5efa
Enable bash linter (#271)
askolesov Jan 31, 2022
bbe75fa
Merge branch 'main' into unified-net-config-generation
askolesov Feb 9, 2022
0b8ad06
Fix step name
askolesov Feb 9, 2022
e039fd7
Get rid of files included because of merge conflict
askolesov Feb 9, 2022
85d94bf
Fix build-tar, build-deb scripts
askolesov Feb 9, 2022
d9646b8
Various merge fixes
askolesov Feb 9, 2022
06db9dc
Fix docker-compose-localnet merge conflicts
askolesov Feb 9, 2022
242e3ba
Fix images build job names
askolesov Feb 9, 2022
55caf39
More merge fixes
askolesov Feb 10, 2022
a4008e2
Fix base image
askolesov Feb 10, 2022
ab697fc
Improve job names
askolesov Feb 10, 2022
118f5e3
Fix test job
askolesov Feb 10, 2022
61d00a3
Restore docker-compose required for the upgrade tests
askolesov Feb 10, 2022
00a7372
Fix docker-compose up
askolesov Feb 10, 2022
a9e6184
Fix broken link
askolesov Feb 10, 2022
dd9d90d
Fix lint issues in common.sh
askolesov Feb 10, 2022
2c9d1b0
Fix linter issues
askolesov Feb 10, 2022
91f8dbb
Fix typo
askolesov Feb 10, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/scripts/ensure_set_euox_pipefail.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,21 @@ set -euo pipefail

INVALID_FILES_FOUND=0

for BASH_SCRIPT in $(find . -type f -name "*.sh")
while IFS= read -r -d '' BASH_SCRIPT
do
echo "checking ${BASH_SCRIPT}"
if ( ! grep -q "set -euo pipefail" "${BASH_SCRIPT}" ) && ( ! grep -q "set -euox pipefail" "${BASH_SCRIPT}" )
then
echo "${BASH_SCRIPT}"
INVALID_FILES_FOUND=1
fi
done
done < <(find . -type f -name "*.sh" -print0) # See https://github.com/koalaman/shellcheck/wiki/SC2044

if [[ INVALID_FILES_FOUND -eq 1 ]]
then
echo ""
echo "The bash scripts above must include either 'set -euo pipefail' or 'set -euox pipefail."
exit 1
else
echo "euox-pipefaile check finished successfully"
fi
1 change: 1 addition & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ jobs:
VALIDATE_XML: true
VALIDATE_YAML: true
VALIDATE_OPENAPI: true
VALIDATE_BASH: true

VALIDATE_ALL_CODEBASE: true
LOG_LEVEL: WARN
Expand Down
131 changes: 61 additions & 70 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,37 +6,12 @@ on:
- "v*"

jobs:
setup-workflow:
name: "Setup workflow"
runs-on: ubuntu-latest
outputs:
GITHUB_REPOSITORY_NAME: ${{ steps.cache.outputs.GITHUB_REPOSITORY_NAME }}
GITHUB_ORG_NAME: ${{ steps.cache.outputs.GITHUB_ORG_NAME }}
TAG: ${{ steps.cache.outputs.TAG }}
VERSION: ${{ steps.cache.outputs.VERSION }}
steps:
- name: Git checkout
uses: actions/checkout@v2

- name: Set outputs
id: cache
run: |
# Exposes org_name in lower case. Required by Docker.
echo "::set-output name=GITHUB_ORG_NAME::$(echo ${GITHUB_REPOSITORY_OWNER,,})"

# Exposes org_name/repository_name in lower case. Required by Docker.
echo "::set-output name=GITHUB_REPOSITORY_NAME::$(echo ${GITHUB_REPOSITORY,,})"

# Version is tag without `v` prefix. Required by Debian package.
TAG=${GITHUB_REF/refs\/tags\//}
echo "::set-output name=TAG::$TAG"

VERSION="${TAG:1}" # Remove first character
echo "::set-output name=VERSION::$VERSION"

build-binary:
name: Build binary
runs-on: ubuntu-20.04
outputs:
VERSION: ${{ steps.set-version.outputs.VERSION }}
steps:
- name: Set up Go 1.17
uses: actions/setup-go@v2
Expand Down Expand Up @@ -81,59 +56,86 @@ jobs:
name: cheqd-noded
path: build/cheqd-noded

build-deb-package:
name: Build Debian package
- name: Set version
id: set-version
run: |
VERSION=$(build/cheqd-noded version 2>&1)
echo "::set-output name=VERSION::$VERSION"

build-tar-package:
name: Build tar package
runs-on: ubuntu-20.04
needs: [ build-binary, setup-workflow ]
needs: build-binary
env:
PACKAGE_NAME: "cheqd-node"
VERSION: ${{ needs.setup-workflow.outputs.VERSION }}
VERSION: ${{ needs.build-binary.outputs.VERSION }}
steps:
- uses: actions/checkout@v2

- name: Load binary artifact
uses: actions/download-artifact@v2
with:
name: cheqd-noded
path: /home/runner/go/bin

- name: Restore binary permissions
run: sudo chmod +x /home/runner/go/bin/cheqd-noded
run: sudo chmod +x cheqd-noded

- name: Make intermediate archive
- name: Build tar
working-directory: ./build-tools
run: |
./build-tar.sh ${{ env.PACKAGE_NAME }} ${{ env.VERSION }}
./build-tar.sh "../cheqd-noded"

- name: Store tar package artifact
uses: actions/upload-artifact@v2
with:
name: cheqd-node_${{ env.VERSION }}.tar.gz
path: build-tools/output/cheqd-node_${{ env.VERSION }}.tar.gz

build-deb-package:
name: Build Debian package
runs-on: ubuntu-20.04
needs: [ build-binary ]
env:
VERSION: ${{ needs.build-binary.outputs.VERSION }}
steps:
- uses: actions/checkout@v2

- name: Load binary artifact
uses: actions/download-artifact@v2
with:
name: cheqd-noded

- name: Restore binary permissions
run: sudo chmod +x cheqd-noded

- name: Install fpm
run: |
sudo apt-get install ruby ruby-dev rubygems build-essential
sudo gem install --no-document fpm

- name: Build Debian package # Depends on tar archive
- name: Build deb
working-directory: ./build-tools
run: |
./build-deb.sh ${{ env.PACKAGE_NAME }} ${{ env.VERSION }}
./build-deb.sh "../cheqd-noded"

- name: Store Debian package artifact
uses: actions/upload-artifact@v2
with:
name: ${{ env.PACKAGE_NAME }}_${{ env.VERSION }}_amd64.deb
path: build-tools/output/${{ env.PACKAGE_NAME }}_${{ env.VERSION }}_amd64.deb
name: cheqd-node_${{ env.VERSION }}_amd64.deb
path: build-tools/output/cheqd-node_${{ env.VERSION }}_amd64.deb

build-node-image:
name: Build cheqd-node Docker image
runs-on: ubuntu-20.04
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0 # FIXME: Starport requires full repository
fetch-depth: 0 # Required to fetch version

- name: Build cheqd-cli with 'cheqd-noded' as entrypoint
run: docker build --target base -t cheqd-cli -f docker/Dockerfile --build-arg UID=$(id -u) --build-arg GID=$(id -g) .
run: docker build --target base -t cheqd-cli -f docker/Dockerfile .

- name: Build cheqd-node with 'node-start' as entrypoint
run: docker build --target node -t cheqd-node -f docker/Dockerfile --build-arg UID=$(id -u) --build-arg GID=$(id -g) .
run: docker build --target node -t cheqd-node -f docker/Dockerfile .

- name: Save cheqd-cli
run: docker save -o cheqd-cli-image.tar cheqd-cli
Expand Down Expand Up @@ -184,15 +186,10 @@ jobs:
publish:
name: "Publish images and release"
# Reference to workflow-setup job is required to access its various outputs.
needs: [ setup-workflow, build-node-image, build-testnet-image, build-deb-package ]
needs: [ build-binary, build-node-image, build-testnet-image, build-deb-package, build-tar-package ]
runs-on: ubuntu-latest
env:
GITHUB_REPOSITORY_NAME: ${{ needs.setup-workflow.outputs.GITHUB_REPOSITORY_NAME }}
GITHUB_ORG_NAME: ${{ needs.setup-workflow.outputs.GITHUB_ORG_NAME }}
TAG: ${{ needs.setup-workflow.outputs.TAG }}
VERSION: ${{ needs.setup-workflow.outputs.VERSION }}
PACKAGE_NAME: "cheqd-node"
TESTNET_IMAGE_NAME: "cheqd-testnet"
VERSION: ${{ needs.build-binary.outputs.VERSION }}
steps:
- name: Git checkout
uses: actions/checkout@v2
Expand Down Expand Up @@ -227,10 +224,10 @@ jobs:

- name: Push cheqd-node image
run: |
docker tag cheqd-node ghcr.io/${{ env.GITHUB_REPOSITORY_NAME }}:${{ env.VERSION }}
docker tag cheqd-node ghcr.io/${{ env.GITHUB_REPOSITORY_NAME }}:latest
docker push ghcr.io/${{ env.GITHUB_REPOSITORY_NAME }}:${{ env.VERSION }}
docker push ghcr.io/${{ env.GITHUB_REPOSITORY_NAME }}:latest
docker tag cheqd-node ghcr.io/${{ github.repository }}:${{ env.VERSION }}
docker tag cheqd-node ghcr.io/${{ github.repository }}:latest
docker push ghcr.io/${{ github.repository }}:${{ env.VERSION }}
docker push ghcr.io/${{ github.repository }}:latest


- name: Push cheqd-cli image
Expand All @@ -242,32 +239,26 @@ jobs:

- name: Push testnet image
run: |
docker tag cheqd-testnet ghcr.io/${{ github.repository_owner }}/${{ env.TESTNET_IMAGE_NAME }}:${{ env.VERSION }}
docker tag cheqd-testnet ghcr.io/${{ github.repository_owner }}/${{ env.TESTNET_IMAGE_NAME }}:latest
docker push ghcr.io/${{ github.repository_owner }}/${{ env.TESTNET_IMAGE_NAME }}:${{ env.VERSION }}
docker push ghcr.io/${{ github.repository_owner }}/${{ env.TESTNET_IMAGE_NAME }}:latest
docker tag cheqd-testnet ghcr.io/${{ github.repository_owner }}/cheqd-testnet:${{ env.VERSION }}
docker tag cheqd-testnet ghcr.io/${{ github.repository_owner }}/cheqd-testnet:latest
docker push ghcr.io/${{ github.repository_owner }}/cheqd-testnet:${{ env.VERSION }}
docker push ghcr.io/${{ github.repository_owner }}/cheqd-testnet:latest
ankurdotb marked this conversation as resolved.
Show resolved Hide resolved

- name: Download deb
uses: actions/download-artifact@v2
with:
name: ${{ env.PACKAGE_NAME }}_${{ env.VERSION }}_amd64.deb
name: cheqd-node_${{ env.VERSION }}_amd64.deb

- name: Load binary artifact
- name: Download tar
uses: actions/download-artifact@v2
with:
name: cheqd-noded

- name: Restore binary permissions
run: sudo chmod +x cheqd-noded

- name: Make tar archive
run: tar -zcvf ${{ env.PACKAGE_NAME }}_${{ env.VERSION }}.tar.gz cheqd-noded
name: cheqd-node_${{ env.VERSION }}.tar.gz

- uses: "marvinpinto/action-automatic-releases@latest"
with:
repo_token: "${{ secrets.GITHUB_TOKEN }}"
automatic_release_tag: "${{ env.TAG }}"
automatic_release_tag: "${{ env.GITHUB_REF_NAME }}"
prerelease: true
files: |
${{ env.PACKAGE_NAME }}_${{ env.VERSION }}_amd64.deb
${{ env.PACKAGE_NAME }}_${{ env.VERSION }}.tar.gz
cheqd-node_${{ env.VERSION }}_amd64.deb
cheqd-node_${{ env.VERSION }}.tar.gz
Loading