Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: GitHub Actions 🚀 #377

Merged
merged 21 commits into from
Dec 8, 2021
Merged

feat: GitHub Actions 🚀 #377

merged 21 commits into from
Dec 8, 2021

Conversation

guidojw
Copy link
Member

@guidojw guidojw commented Nov 23, 2021 •
edited
Loading

Summary

Moves our CI/CD pipelines from Buildkite to GitHub Actions!

Worfklows & explanations

There are four new workflows:

  • Continuous Integration: does the linting and tests. The build workflow uploads the built images to the workflow artifacts so that the other jobs can download the images from there and then load them. The problem matchers here are responsible for the PR Annotations.
  • Continuous Delivery: responsible for deploying staging/master branches to the server. Can merge changes on staging into master first. Makes use of GitHub Deployments and Sentry Releases. It finalizes Sentry releases created in the Publish Image workflow and implicitly creates Sentry deployments for those releases.
  • Publish Image: on pushes to master and staging, publishes images to the GitHub Container Registry. Can also be manually triggered. This workflow will create unfinalized Sentry releases and couple commits & sourcemaps to them.
  • Cleanup Registry: scheduled for every Monday at 00:00, removes old images from the registry.

Some of these workflows use actions from a repository of mine (https://github.com/guidojw/actions), this is because these actions are very generic and I'm gonna use them in a similar manner in amber-api, sofia and my own projects.

Cool new features

  • Staging can be automatically merged into master before deploying to production.
  • PR Annotations, meaning things like linting warnings and errors show up in the "Files changed" tab on PRs.
  • Deployments of the environments show on the repository page.
  • Sentry records commits with releases (so we get cool things like suspect commits on issues).
  • Sourcemaps are uploaded with Sentry releases, so Sentry can couple errors in the minified JavaScript on clients to the actual source code.
  • Sentry deployments are used.

Quirks

  • I originally had set a concurrency group for the Continuous Integration workflow so that only one run is possible per branch and new runs cancel old ones. This hit a limitation of reusable workflows where they cannot set concurrency, so it's removed now.
  • I tried to DRY up the update check run jobs in the Publish Image and Continuous Delivery workflows by moving it to a reusable workflow, but GitHub Actions has a limitation that calling reusable workflows can only go 1 call deep, so this can't work now :(.
  • Stylelint's output isn't compatible with GitHub Actions problem matchers (used for PR Annotations), so I hacked together a Node.js script that converts it.
  • Getting sourcemaps in the Publish Image workflow requires pulling the just pushed image from the registry first. I haven't found a feasible way to not have to do this yet.
  • yes (contexts don't support ternaries).
  • We will not get build notifications on the Slack anymore, but check statuses are included in the bottom of PR embeds in #repositories so I figured this is not that big of a problem.

TODO:

  • Create internal app "Alpha Releases" on Sentry with org:read and releases:admin and add auth token to repository secrets named SENTRY_AUTH_TOKEN.
  • Rename amber-ui-production on Sentry to amber-ui?
  • Disable amber-ui pipeline on Buildkite.
  • Change docker-compose.yml on the server to point to image on ghcr.io.
  • Change branch protection rules to exclude the Buildkite check and include these.

Other information

@guidojw
feat(CI): Continuous Integration
0a80301 
fix(CI): remove old maintainer command

feat(CI): try setting maintainer anothe rway

feat(CI): use load Docker image action from own repository

feat(CI): remove actions directory

feat(CI): link action to own repository

test(CI): test caching

fix(CI): correct variable interpolation

test(CI): test caching

feat(CI): scope cache by file

test(CI): test caching

fix(CI): also remove scope in publish job

test(CI): remove scope

fix(CI): undo ESLint matcher trigger test

fix(Dockerfile): move maintainer label to top stage

fix(CI): typo in variable name

refactor(CI): rename Dockerfile.ci

fix(CI): undo stylelint trigger test

fix(CI): correct path to stylelint converter

fix(CI): run both commands and fail if one fails

feat(CI): retry stylelint problem matcher

fix(CI): try fixing path

fix(CI): try without tty

fix(CI): revert stylelint trigger test changes

test(CI): trigger ESLint

fix(CI): don't add styleling matcher

fix(CI): remove stylelint matcher related code

fix(CI): try other path

fix(CI): try fixing path

fix(CI): change stylelint command to path

fix(CI): set npm path

test(CI): convert stylelint output in bin

feat(CI): add stylelint problem matcher

fix(CI): checkout code first

fix(CI): try wrapping path in quotes

fix(CI): change add matcher command

feat(CI): use problem matchers instead

fix(CI): remove fail-on-error

test(CI): remove level

fix(CI): readd filter mode and change level to warning

fix(CI): remove filter mode

test(CI): test if reviewdog works

feat(CI): change reviewdog filter to no filter

fix(CI): add checkout code action to hopefully fix reviewdog error

fix(CI): put eslint sed pattern in quotes

fix(CI): use sed again for eslint output

fix(CI): use grep to find rdjson output

fix(CI): try removing footer line with sed too

fix(CI): simplify sed commands

fix(CI): try scripts again but remove lines first

fix(CI): call lint commands directly

fix(CI): rename stylelint reviewdog

fix(CI): decouple stylelint from lint command

fix(CI): remove interactive terminal to try fix reviewdog

fix(CI): try fixing lint command for ci in package

feat(CI): setup reviewdog

fix(CI): also run on PR synchronize

test(CI): test Slack webhook

test(CI): test Slack webhook

test(CI): test webhook

feat(CI): uncomment ref check for production image build

fix(CI): remove cache mode

fix(CI): also color test output

feat(CI): try setting color

feat(CI): try other cache mode

fix(CI): fix image tag

refactor(CI): remove metadata action

fix(CI): typo in login action version

feat(CI): use metadata action

fix(CI): yml syntax error

feat(CI): try publishing to GHCR

fix(CI): test cache scopes

fix(CI): scope cache

feat(CI): add concurrency settings

feat(CI): make upload optional

fix(CI): fix if expression II

fix(CI): fix if expression

fix(CI): change built app name

feat(CI): test building staging image

feat(CI): support build args and add publish mock

refactor(CI): move building to composite action

feat(CI): add test step

fix(CI): add tags to build-push action

fix(CI): test if Docker Buildx fixes image loading

fix(CI): correct image ls command

fix(CI): try printing images

fix(CI): use correct flag for docker load

fix(CI): fix action and change output type

feat(CI): test image loading

feat(CI): test caching

fix(CI): build Dockerfile

feat(CI): test image building and caching using Buildx

feat(CI): actually run lint and add test

fix: remove emojis from step name

refactor: rename docker-compose.buildkite.yml

fix(CI): move checkout code out of step

feat(CI): test Docker image building

feat: add file

refactor(CI): move problem matchers to own folder

refactor(CI): update workflow file

refactor(CI): move problem matchers to own folder

refactor(CI): update workflow file
@guidojw
feat(CD): Continuous Delivery
b47c16b 
feat(CD): add workflow

fix(CI): read correct event input and fix if

fix(CD): point to feature branch

feat(CD): begin implementation

test(CI): more ifs tests

test(CI): test ifs

test(CI): try other status check

test(CI): test needs + if

feat(CI): add publish image workflow

refactor: update publish image workflow file

feat(CI): add publish image workflow

test(CI): test needs + if

test(CI): try other status check

test(CI): test ifs

test(CI): more ifs tests

feat(CD): begin implementation

fix(CD): point to feature branch

fix(CI): read correct event input and fix if

feat(CD): add workflow

refactor: update publish image workflow file
@guidojw
chore: update README
096bf02 
chore: add badges to README

chore: update README

chore: add badges to README
@guidojw
refactor: remove Buildkite configurations
6769a5d 
chore: remove last Buildkite occurences from code

refactor: remove .buildkite directory

chore: remove last Buildkite occurences from code
@guidojw
feat: add workflow for cleaning up registry
02ee656 
refactor: use repository owner from context instead of static

feat: add workflow for cleaning up registry

refactor: use repository owner from context instead of static
@guidojw guidojw marked this pull request as ready for review November 23, 2021 23:54
@guidojw guidojw changed the title feat: GitHub Actions feat: GitHub Actions 🚀 Nov 24, 2021
@guidojw guidojw merged commit 39e6993 into csvalpha:staging Dec 8, 2021
@guidojw guidojw deleted the feat/github-actions-ci branch December 8, 2021 23:18
@guidojw guidojw mentioned this pull request Feb 13, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wilco375 wilco375 wilco375 approved these changes

Assignees
No one assigned
Labels
type:feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@guidojw @wilco375