Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify script for deployment #398

Merged
merged 7 commits into from
Mar 12, 2022
Merged

Modify script for deployment #398

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8504650
Convert to resource-agnostic version
arturtamborski May 8, 2021
5b44354
Extend travis build to deploy to production on tag push
arturtamborski May 9, 2021
c616a92
typo
arturtamborski May 9, 2021
aa0c02b
Add deployment env variables
arturtamborski May 9, 2021
76be017
Use deployment env variables
arturtamborski May 9, 2021
ca2c1e8
make variable names more consistent
arturtamborski May 9, 2021
efd6831
Merge branches 'master' and '385-prod-deploy' of github.com:arturtamb…
arturtamborski Mar 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 28 additions & 8 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,25 @@
os: linux
dist: xenial
language: python
python: 3.8

env:
global:
- SECRET_KEY=codeforpoznan
# AWS_ACCESS_KEY_ID
- AWS_REGION=eu-west-1

- STAGING_RESOURCE=dev_codeforpoznan_pl_v3
# STAGING_AWS_ACCESS_KEY_ID
- secure: "lP1qLhUyJgoJlDqva3dYq3jQh8j1jSXJx19Lq4oNwMLxk4d567LAA/AZHBj8lRqrwW/tjiONb5o1V15+9ogfoK7Y8ika7F3YZkUQ5k/OOliQzDqditSWjYCv3Rx6fc1D5UGQLF1MYYjGctDxiRHmPWb2WfIoqUdrcN4GAbXBKA/EGn8Y680JZm6XZkXFc2PjqoILPwJ+ZIWqAIELibnpHPnRDHD28a3uXjWa1Syehctj2f44wxRNo1hJIQp2mlO9BFS2LrCrhUD2OgeC8gy/j5itzNcDrRwnwZmeZ92YdSsNGGark3K3zPyl5em5v7zlJB/6f/6Jf1IIU0hpc1QAUwZjPd7rRBeYR2U8vUBdYrBlhKFwAJjQjGj4Xm0dgIjcFiANtfppxMt7cFg/c9AUkI+jLGdqfGxkVYKtvYAYWC6QEDikKFwz8eN94zfmkMRkXcniSZPkXAuwLEl9ozz+QAzzNs8BzxzMV4pFB+N/of88B91FyWCchxzSftbrgc8wFmMFZGwzfStgbDU4RX3sGeY7af7rF6uCYS6eHEVwllvJefyNkjUIR8x7wmCXaPjHnKgRzuYrcLtTNUxuc0LMfLCxqVCkGXxflQNIY7Zd59SWukJyloZbUZhxYuoX7YTwVHynNrx+pxvPBP/0QaSnuT8RBZOxmDf1HqT1ShvQgsg="
# AWS_SECRET_ACCESS_KEY
# STAGING_AWS_SECRET_ACCESS_KEY
- secure: "JggIiR6ZzXMoK7uCxMgB4nkPzWl7aYXTvIUlkjDdt0OHHySMAEv1D6m45npjCYTqSXwiEuNSplRCMvK72Br1PfWl6hu3d+Nr6DxDTjVu0Xs2jW55h5+A+VNS/Df7Rb2cQ7Ei9e+KI2yoD5oF4ayp62P0RXu060f/9jVnZNFHmR0yJ97eZLcSuY2nmxfQXK/ADnhKIOg0wk2MEnKPlVjzfFBFhvM7h+fmWGaD02QqBfijfnbH6vRdEWICvdkY3eE0Ah7fnETEYMbGVdzGFBouFx66BRKcBFZPbkRtfDNbkIjiClQjWa/HpeXjNxSfgdVrse826qBUN2FrUMR1lxUq4lzSJqNBEQq12if19RFaI12grLo7zfgDIUTOCgXtR+9hGrFredweE7E+q4SmmeFKrzI3fElnEq3PjCzOMMYkF0u3Fvhm9yncZ52SiZYovF4ws6FNxlNud6t6u4jgvphr6fUAHq94g/lLfPgxM/LEIfCaXYHAQ1bKH9MGdqP8zTmQC0OQ2QcpUFl961fiUowIawGrUnkddQGB1AsVlIYehMhVYIhvV7DI9X9ZpYbbADS9WKy2FTzD2KezuJrmZHUumTVT22ZMGai0wP/deLaNEklT6p5qgid4d+ifSvOwaBSTghqpktE6/DbkcgAx4I8o8RkGsl6kucdy8USCRYTHWAE="

- PRODUCTION_RESOURCE=codeforpoznan_pl_v3
# PRODUCTION_AWS_ACCESS_KEY_ID
#- secure: "TODO REPLACE ME WITH ENCRYPTED VALUES"
# PRODUCTION_AWS_SECRET_ACCESS_KEY
#- secure: "TODO REPLACE ME WITH ENCRYPTED VALUES"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

waiting for output from CodeForPoznan/infrastructure#84


install:
- pip install awscli
- nvm install 10.13.0
Expand All @@ -21,9 +32,18 @@ script:
- (cd backend && black --check .) # lint backend

deploy:
skip_cleanup: true
provider: script
script: bash deploy.sh
on:
branch: master
repo: CodeForPoznan/codeforpoznan.pl_v3
# staging -- https://dev.codeforpoznan.pl
- provider: script
skip_cleanup: true
script: bash deploy.sh staging
on:
branch: master
repo: CodeForPoznan/codeforpoznan.pl_v3

# production -- https://codeforpoznan.pl
- provider: script
skip_cleanup: true
script: bash deploy.sh production
on:
tags: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how we going to control releases, isn't it?

repo: CodeForPoznan/codeforpoznan.pl_v3
85 changes: 61 additions & 24 deletions deploy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,41 +1,77 @@
#!/usr/bin/env bash
set -Eeuo pipefail


Comment on lines +3 to +4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some cases you use singular and in some double breaklines and I presume you did it on purpose. If it matters for some reason maybe it would be worth to leave //some kind of docstring?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's just a visual separation of bits in script file. Most of the actions in the script are doubled (one for frontend and then right after one for backend, or one for api lambda and right after one for migration lambda) - so the double newline is just to visually help recognize particular 'steps' in script.

case "${1:-}" in
staging)
echo "deploying to staging"
export RESOURCE="${STAGING_RESOURCE}"
export AWS_ACCESS_KEY_ID="${STAGING_AWS_ACCESS_KEY_ID}"
export AWS_SECRET_ACCESS_KEY="${STAGING_AWS_SECRET_ACCESS_KEY}"
;;

production)
echo "deploying to production"
export RESOURCE="${PRODUCTION_RESOURCE}"
export AWS_ACCESS_KEY_ID="${PRODUCTION_AWS_ACCESS_KEY_ID}"
export AWS_SECRET_ACCESS_KEY="${PRODUCTION_AWS_SECRET_ACCESS_KEY}"
;;

*)
echo "invalid environment '${1}', exiting..."
exit 1
;;
esac


echo "build and push frontend"
(cd frontend && yarn run build && cp -r dist ../public)
aws s3 sync --delete public s3://codeforpoznan-public/dev_codeforpoznan_pl_v3
aws cloudfront create-invalidation --paths "/*" --distribution-id E6PZCV3N5WWJ8
aws s3 sync --delete public "s3://codeforpoznan-public/${RESOURCE}"


echo "refresh CDN"
dist_id=$(
aws cloudfront list-distributions \
| jq --arg PATH "/${RESOURCE}" -r '
.DistributionList.Items[]
| {Id, p: .Origins.Items[].OriginPath}
| select(.p == $PATH).Id'
)
aws cloudfront create-invalidation --paths "/*" --distribution-id "${dist_id}"


echo "bundle application"
(cd backend && pipenv run pip install -r <(pipenv lock -r) --target ../packages)
(cd backend && pipenv run pip install -r <(pipenv lock -r) --target ../packages)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this second whitespace is here? Mistake or on purpose?

Copy link
Member Author

@arturtamborski arturtamborski Mar 12, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just to make it easier to the eye - I like aligned lines, so it' easier to compare visually differences between two similar ones.

There's a lot of similar one-liners in the script as you can see, and the differences are mostly tiny, so by aligning the characters in key points (like here, on operator '&') I hope to make it more clear which bits are different.

(cd packages && zip -qgr9 ../lambda.zip .)
ln -s backend/migrations migrations
ln --symbolic backend/migrations migrations
zip --symlinks -qgr9 lambda.zip backend/ migrations/


echo "upload lambdas"
aws s3 cp lambda.zip s3://codeforpoznan-lambdas/dev_codeforpoznan_pl_v3_serverless_api.zip
aws s3 cp lambda.zip s3://codeforpoznan-lambdas/dev_codeforpoznan_pl_v3_migration.zip
aws s3 cp lambda.zip "s3://codeforpoznan-lambdas/${RESOURCE}_serverless_api.zip"
aws s3 cp lambda.zip "s3://codeforpoznan-lambdas/${RESOURCE}_migration.zip"


echo "refresh lambdas"
aws lambda update-function-code \
--function-name dev_codeforpoznan_pl_v3_serverless_api \
--s3-bucket codeforpoznan-lambdas \
--s3-key dev_codeforpoznan_pl_v3_serverless_api.zip \
--region eu-west-1 \
| jq 'del(.Environment, .VpcConfig, .Role, .FunctionArn)' \

aws lambda update-function-code \
--function-name dev_codeforpoznan_pl_v3_migration \
--s3-bucket codeforpoznan-lambdas \
--s3-key dev_codeforpoznan_pl_v3_migration.zip \
--region eu-west-1 \
| jq 'del(.Environment, .VpcConfig, .Role, .FunctionArn)' \
aws lambda update-function-code \
--s3-bucket "codeforpoznan-lambdas" \
--s3-key "${RESOURCE}_serverless_api.zip" \
--function-name "${RESOURCE}_serverless_api" \
| jq 'del(.Environment, .VpcConfig, .Role, .FunctionArn)' \

aws lambda update-function-code \
--s3-bucket "codeforpoznan-lambdas" \
--s3-key "${RESOURCE}_migration.zip" \
--function-name "${RESOURCE}_migration" \
| jq 'del(.Environment, .VpcConfig, .Role, .FunctionArn)' \


echo "run migrations"
aws lambda invoke \
--function-name dev_codeforpoznan_pl_v3_migration \
--region eu-west-1 \
response.json \
> request.json \
aws lambda invoke \
--function-name "${RESOURCE}_migration" \
response.json \
> request.json \


echo "show migration output"
jq -s add ./*.json | jq -re '
Expand All @@ -45,4 +81,5 @@ jq -s add ./*.json | jq -re '
.stdout, .stderr
end'


exit $?