chore(build): switch to github workflows for robot stack CI #6632
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| name: 'Deploy wheel to pypi' | ||
| description: 'Deploy a python wheel to pypi or test pypi, depending on configuration. The python environment for the subproject must be set up already, and the complex environment variables must be defined.' | ||
| inputs: | ||
| project: | ||
| description: 'Which project to run make deploy in' | ||
| required: true | ||
| repository_url: | ||
| description: 'The repository url to upload to. Creds will be determined based on this' | ||
| required: true | ||
| password: | ||
| description: 'The repository password' | ||
| runs: | ||
| using: 'composite' | ||
| steps: | ||
| - shell: bash | ||
| run: | | ||
| echo "::add-mask::${{ inputs.password }}" | ||
| if [[ ${{ inputs.repository_url }} =~ "test" ]]; then | ||
| echo '::set-env name=SWALLOW_FAILURE::|| echo "Duplicate upload allowed on test"' | ||
| echo "Uploading to test pypi" | ||
| else if [[ ${{ inputs.repository_url }} =~ "upload.pypi.org" ]]; then | ||
| echo "Uploading to prod pypi" | ||
| # this needs to be cleared otherwise the makefiles append a dev tag | ||
| echo ::set-env name=OT_BUILD::"" | ||
| else | ||
| echo "::error ::Invalid repository url ${{ inputs.repository_url }}" | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
|
|
||
| - shell: bash | ||
| run: | | ||
| status=0 | ||
| QUIET=1 BUILD_NUMBER=${OT_BUILD} make -C ${{ inputs.project }} clean deploy twine_repository_url=${{ inputs.repository_url }} pypi_username=opentrons pypi_password=${{ inputs.password }} || status=$? | ||
| if [[ ${status} != 0 ]] && [[ ${{ inputs.repository_url }} =~ "test.pypi.org" ]]; then | ||
| echo "upload failures allowed to test pypi" | ||
| exit 0 | ||
| fi | ||
| exit ${status} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,142 @@ | ||
| # Run tests, build the app, and deploy it cross platform | ||
|
|
||
| name: 'App test, build, and deploy' | ||
|
|
||
| on: | ||
| push: | ||
| paths: | ||
| - 'app/**/*' | ||
| - 'app-shell/**/*' | ||
| - 'components/**/*' | ||
| - 'shared-data/**/*' | ||
| - 'webpack-config/**/*' | ||
| - 'discovery-client/**/*' | ||
| - '*.js' | ||
| - 'scripts/**/*' | ||
| - '*.json' | ||
| - 'yarn.lock' | ||
| - '.github/workflows/app-test-build.yaml' | ||
| - '.github/workflows/utils.js' | ||
| branches: | ||
| - '*' | ||
| tags: | ||
| - 'v*' | ||
| pull_request: | ||
| paths: | ||
| - 'app/**/*' | ||
| - 'app-shell/**/*' | ||
| - 'components/**/*' | ||
| - 'shared-data/**/*' | ||
| - 'webpack-config/**/*' | ||
| - 'discovery-client/**/*' | ||
| - '*.js' | ||
| - '*.json' | ||
| - 'yarn.lock' | ||
| - 'scripts/**/*' | ||
| workflow_dispatch: | ||
|
|
||
| #defaults: | ||
| # run: | ||
| # shell: bash | ||
|
|
||
| env: | ||
| CI: true | ||
| OT_APP_DEPLOY_BUCKET: opentrons-app | ||
| OT_APP_DEPLOY_FOLDER: builds-actions # Not the actual deploy target - change when we cut over | ||
|
|
||
| jobs: | ||
| js-unit-test: | ||
| # unit tests for js frontend projects (e.g. not app-shell or discovery-client) do not need | ||
| # to run cross-platform | ||
| runs-on: 'ubuntu-latest' | ||
| name: 'opentrons app frontend unit tests' | ||
| steps: | ||
| - uses: 'actions/checkout@v2' | ||
| - uses: 'actions/setup-node@v1' | ||
| with: | ||
| node-version: '12' | ||
| - name: 'set complex environment variables and get yarn cache' | ||
| id: 'set-vars-get-cache' | ||
| uses: actions/github-script@v2 | ||
| with: | ||
| script: | | ||
| const { buildComplexEnvVars, findYarnCacheDir } = require(`${process.env.GITHUB_WORKSPACE}/.github/workflows/utils.js`) | ||
| buildComplexEnvVars(core, context) | ||
| findYarnCacheDir(core, context) | ||
mcous marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - name: 'cache yarn cache' | ||
| uses: actions/cache@v2 | ||
| with: | ||
| path: ${{ steps.set-vars-get-cache.outputs.yarnCacheDir }} | ||
| key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-yarn- | ||
|
There was a problem hiding this comment. Out of curiosity, instead of grabbing the global yarn cache from your util, why not just have a step that sets the cache directory like here? Also, I don't see where behavior changes when there's a cache hit, you're still going to do a There was a problem hiding this comment. Yeah but that should prevent yarn from redownloading everything shouldn't it? In terms of why put it in a file, it accomplishes essentially the same thing and the file really had to be there for the custom env vars so I built it in anyway There was a problem hiding this comment.
Ah yeah that makes sense. Is it working as expected now? Speaking of which, how are you validating that this works? I assume just looking through the action logs and making sure yarn doesn't redownload everything? There was a problem hiding this comment. Nvm, can see it's working now. Nice! https://github.com/Opentrons/opentrons/runs/1193602613?check_suite_focus=true |
||
| - name: 'setup-js' | ||
| run: 'make setup-js' | ||
| - name: 'test frontend packages' | ||
| run: | | ||
| yarn jest --coverage=true --ci=true --testPathIgnorePatterns 'labware-.*' 'protocol-.*' 'app-shell' 'discovery-client' | ||
|
|
||
| build-app-test-backend: | ||
| # since js tests for "backend" projects (app-shell, discovery-client) need | ||
| # to run cross-platform just like builds, might as well do them in the same job | ||
| strategy: | ||
| matrix: | ||
| os: ['windows-latest', 'ubuntu-latest', 'macos-latest'] | ||
| name: 'opentrons app backend unit tests and build' | ||
| runs-on: ${{ matrix.os }} | ||
| steps: | ||
| - uses: 'actions/checkout@v2' | ||
| - uses: 'actions/setup-node@v1' | ||
| with: | ||
| node-version: '12' | ||
| - name: 'set complex environment variables and get yarn cache' | ||
| id: 'set-vars-get-cache' | ||
| uses: actions/github-script@v2 | ||
| with: | ||
| script: | | ||
| const { buildComplexEnvVars, findYarnCacheDir } = require(`${process.env.GITHUB_WORKSPACE}/.github/workflows/utils.js`) | ||
| buildComplexEnvVars(core, context) | ||
| findYarnCacheDir(core, context) | ||
| - name: 'cache yarn cache' | ||
mcous marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| uses: actions/cache@v2 | ||
| with: | ||
| path: ${{ steps.set-vars-get-cache.outputs.yarnCacheDir }} | ||
| key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-yarn- | ||
| - name: 'setup-js' | ||
| run: 'make setup-js' | ||
| - name: 'test native(er) packages' | ||
| run: | | ||
| yarn jest --coverage=true --ci=true --testPathPattern '(app-shell|discovery-client)' | ||
| - if: contains(matrix.os, 'ubuntu') | ||
mcous marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| uses: 'codecov/codecov-action@v1' | ||
| with: | ||
| file: coverage.xml | ||
mcous marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| # build the app and deploy it | ||
| - if: github.event_name != 'pull_request' | ||
| name: 'build app for ${{ matrix.os }}' | ||
mcous marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| env: | ||
| OT_APP_MIXPANEL_ID: ${{ secrets.OT_APP_MIXPANEL_ID }} | ||
| OT_APP_INTERCOM_ID: ${{ secrets.OT_APP_INTERCOM_ID }} | ||
| WIN_CSC_LINK: ${{ secrets.OT_APP_CSC_WINDOWS }} | ||
| WIN_CSC_KEY_PASSWORD: ${{ secrets.OT_APP_CSC_KEY_WINDOWS }} | ||
| CSC_LINK: ${{ secrets.OT_APP_CSC_MACOS }} | ||
| CSC_KEY_PASSWORD: ${{ secrets.OT_APP_CSC_KEY_MACOS }} | ||
| APPLE_ID: ${{ secrets.OT_APP_APPLE_ID }} | ||
| APPLE_ID_PASSWORD: ${{ secrets.OT_APP_APPLE_ID_PASSWORD }} | ||
| run: | | ||
| make -C app-shell dist-${{ matrix.os }} | ||
| - if: github.event_name != 'pull_request' | ||
| env: | ||
| AWS_ACCESS_KEY_ID: ${{ secrets.S3_APP_DEPLOY_KEY_ID }} | ||
| AWS_SECRET_ACCESS_KEY: ${{ secrets.S3_APP_DEPLOY_SECRET }} | ||
| AWS_DEFAULT_REGION: us-east-2 | ||
| name: 'deploy' | ||
| run: aws s3 sync --acl=public-read app-shell/dist/publish s3://${{ env.OT_APP_DEPLOY_BUCKET }}/${{ env.OT_APP_DEPLOY_FOLDER }} | ||
| - if: github.event_name != 'pull_request' | ||
| name: 'upload github artifact' | ||
| uses: actions/upload-artifact@v2 | ||
| with: | ||
| name: 'opentrons app for ${{ matrix.os }}' | ||
| path: app-shell/dist/publish | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Prior to switching PD, we will need to confirm that this is successfully uploading dotfiles in
distPathto S3, which is behavior PD relies on