Skip to content

Commit

Permalink
ARROW-17621: [CI] Audit workflows (apache#14155)
Browse files Browse the repository at this point in the history
In this PR I:
- reduced the scope of the automatically generated `GITHUB_TOKEN` as much as possible (technically `contents:none` would  be the minimum but it is a bit unintuitive as it does not prevent checkout of public repos, I set `contents:read` in those cases)  
- update all actions used to the newest version (checking for breaking changes, only case is actions/github-script which remains on v3 for that reason -> follow up)
- move the creation of envvars containing secrets as close to their usage as possible (-> the step they are used in), this duplicates them in workflows with multiple jobs but is safer.

I have opted **NOT** to pin the different actions by SHA as recommended in some places as the con outweigh the possible protection in my opinion. The main danger with pinning tags or branches is that a malicious actor changes the commit the tag points to and exfiltrates secrets (either repository secrets or in case of private repos code/ip) or takes some other damaging action like deleting branches, rewriting history etc..

We only ever pass actions the `GITHUB_TOKEN` which is ephemeral (deleted after workflow is finished) and scope limited so exfiltration of that token would worst case allow an attacker to create/delete labels and pr comments as well as modify PR branches (if the submitter activated the checkbox for maintainer access). Actions can not access secrets without the workflow author explicitly passing them as input (envvars might reveal them though)

The Apache Org limits the actions that can be used in repos, so we only use well known allow-listed actions, while this does of course not prevent malicious actions it reduces the risk substantially.

Pinning SHAs would mitigate these risks (provided the action at that sha was audited...) but would also necessitate regularly checking + re-auditing the actions as to not miss security patches in these actions (e.g. [here](https://github.com/matlab-actions/setup-matlab/releases/tag/v1.1.1)). IMHO that would be a considerable effort (+ needing real expertise in typescript/node to spot any malicious additions outside of blatant secret exfiltration or nuking) resulting in a small gain.


Lead-authored-by: Jacob Wujciak-Jens <[email protected]>
Co-authored-by: assignUser <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
  • Loading branch information
assignUser authored and zagto committed Oct 7, 2022
1 parent d21c776 commit 32f2665
Show file tree
Hide file tree
Showing 19 changed files with 193 additions and 82 deletions.
3 changes: 3 additions & 0 deletions .github/workflows/archery.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ concurrency:
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
cancel-in-progress: true

permissions:
contents: read

jobs:

test:
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/comment_bot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ on:
- created
- edited

permissions:
contents: read
pull-requests: write

jobs:
crossbow:
name: Listen!
Expand Down
25 changes: 14 additions & 11 deletions .github/workflows/cpp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,12 @@ concurrency:
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
cancel-in-progress: true

permissions:
contents: read

env:
ARROW_ENABLE_TIMING_TESTS: OFF
DOCKER_VOLUME_PREFIX: ".docker/"
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}

jobs:
docker:
Expand All @@ -75,7 +76,7 @@ jobs:
fetch-depth: 0
submodules: recursive
- name: Cache Docker Volumes
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: .docker
key: ${{ matrix.image }}-${{ hashFiles('cpp/**') }}
Expand All @@ -87,12 +88,18 @@ jobs:
- name: Setup Archery
run: pip install -e dev/archery[docker]
- name: Execute Docker Build
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
sudo sysctl -w kernel.core_pattern="core.%e.%p"
ulimit -c unlimited
archery docker run ${{ matrix.image }}
- name: Docker Push
if: success() && github.event_name == 'push' && github.repository == 'apache/arrow'
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
continue-on-error: true
run: archery docker push ${{ matrix.image }}

Expand All @@ -101,8 +108,6 @@ jobs:
runs-on: ubuntu-latest
if: ${{ !contains(github.event.pull_request.title, 'WIP') }}
timeout-minutes: 45
strategy:
fail-fast: false
steps:
- name: Checkout Arrow
uses: actions/checkout@v3
Expand All @@ -123,8 +128,6 @@ jobs:
runs-on: macos-latest
if: ${{ !contains(github.event.pull_request.title, 'WIP') }}
timeout-minutes: 75
strategy:
fail-fast: false
env:
ARROW_BUILD_TESTS: ON
ARROW_DATASET: ON
Expand Down Expand Up @@ -173,7 +176,7 @@ jobs:
run: |
echo "::set-output name=cache-dir::$(ccache --get-config cache_dir)"
- name: Cache ccache
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: ${{ steps.ccache-info.outputs.cache-dir }}
key: cpp-ccache-macos-${{ hashFiles('cpp/**') }}
Expand Down Expand Up @@ -271,7 +274,7 @@ jobs:
run: |
echo "::set-output name=cache-dir::$(ccache --get-config cache_dir)"
- name: Cache ccache
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: ${{ steps.ccache-info.outputs.cache-dir }}
key: cpp-ccache-windows-${{ env.CACHE_VERSION }}-${{ hashFiles('cpp/**') }}
Expand Down Expand Up @@ -357,7 +360,7 @@ jobs:
shell: msys2 {0}
run: ci/scripts/msys2_setup.sh cpp
- name: Cache ccache
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: ccache
key: cpp-ccache-mingw${{ matrix.mingw-n-bits }}-${{ hashFiles('cpp/**') }}
Expand All @@ -379,7 +382,7 @@ jobs:
https://dl.min.io/server/minio/release/windows-amd64/archive/minio.RELEASE.2022-05-26T05-48-41Z
chmod +x /usr/local/bin/minio.exe
- name: Cache Python wheels
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: "${{ env.PIP_CACHE_DIR }}"
key: cpp-wheels-mingw${{ matrix.mingw-n-bits }}-${{ hashFiles('ci/scripts/install_gcs_testbench.sh') }}
Expand Down
9 changes: 6 additions & 3 deletions .github/workflows/csharp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ concurrency:
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
cancel-in-progress: true

permissions:
contents: read

jobs:

ubuntu:
Expand All @@ -46,7 +49,7 @@ jobs:
dotnet: ['6.0.x']
steps:
- name: Install C#
uses: actions/setup-dotnet@v1
uses: actions/setup-dotnet@v2
with:
dotnet-version: ${{ matrix.dotnet }}
- name: Checkout Arrow
Expand Down Expand Up @@ -74,7 +77,7 @@ jobs:
dotnet: ['6.0.x']
steps:
- name: Install C#
uses: actions/setup-dotnet@v1
uses: actions/setup-dotnet@v2
with:
dotnet-version: ${{ matrix.dotnet }}
- name: Checkout Arrow
Expand All @@ -101,7 +104,7 @@ jobs:
dotnet: ['6.0.x']
steps:
- name: Install C#
uses: actions/setup-dotnet@v1
uses: actions/setup-dotnet@v2
with:
dotnet-version: ${{ matrix.dotnet }}
- name: Checkout Arrow
Expand Down
11 changes: 8 additions & 3 deletions .github/workflows/dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,8 @@ concurrency:
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
cancel-in-progress: true

env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
permissions:
contents: read

jobs:

Expand All @@ -48,12 +47,18 @@ jobs:
- name: Setup Archery
run: pip install -e dev/archery[docker]
- name: Execute Docker Build
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
run: |
sudo sysctl -w kernel.core_pattern="core.%e.%p"
ulimit -c unlimited
archery docker run -e GITHUB_ACTIONS=true ubuntu-lint
- name: Docker Push
if: success() && github.event_name == 'push' && github.repository == 'apache/arrow'
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
continue-on-error: true
run: archery docker push ubuntu-lint

Expand Down
31 changes: 16 additions & 15 deletions .github/workflows/dev_pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,14 @@ on:
- edited
- synchronize

# NOTE: not using the "cancel-in-progress" feature here as the group key
# does not have enough information for linking it to a particular PR
concurrency:
group: ${{ github.workflow }}-${{ github.repository }}-${{ github.event.number }}
cancel-in-progress: true

permissions:
contents: read
pull-requests: write

jobs:
process:
name: Process
Expand All @@ -41,9 +46,8 @@ jobs:

- name: Comment JIRA link
if: |
github.event_name == 'pull_request_target' &&
(github.event.action == 'opened' ||
github.event.action == 'edited')
(github.event.action == 'opened' ||
github.event.action == 'edited')
uses: actions/github-script@v3
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -53,9 +57,8 @@ jobs:
- name: Check title
if: |
github.event_name == 'pull_request_target' &&
(github.event.action == 'opened' ||
github.event.action == 'edited')
(github.event.action == 'opened' ||
github.event.action == 'edited')
uses: actions/github-script@v3
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -65,9 +68,8 @@ jobs:
- name: Check Jira Issue
if: |
github.event_name == 'pull_request_target' &&
(github.event.action == 'opened' ||
github.event.action == 'edited')
(github.event.action == 'opened' ||
github.event.action == 'edited')
uses: actions/github-script@v3
with:
debug: true
Expand All @@ -78,10 +80,9 @@ jobs:
- name: Assign GitHub labels
if: |
github.event_name == 'pull_request_target' &&
(github.event.action == 'opened' ||
github.event.action == 'synchronize')
uses: actions/[email protected]
(github.event.action == 'opened' ||
github.event.action == 'synchronize')
uses: actions/labeler@4
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
configuration-path: .github/workflows/dev_pr/labeler.yml
Expand Down
13 changes: 10 additions & 3 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,12 @@ name: Docs
on:
push:

permissions:
contents: read

env:
ARROW_ENABLE_TIMING_TESTS: OFF
DOCKER_VOLUME_PREFIX: ".docker/"
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}

jobs:

Expand All @@ -41,7 +42,7 @@ jobs:
with:
fetch-depth: 0
- name: Cache Docker Volumes
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: .docker
key: ubuntu-docs-${{ hashFiles('cpp/**') }}
Expand All @@ -53,8 +54,14 @@ jobs:
- name: Setup Archery
run: pip install -e dev/archery[docker]
- name: Execute Docker Build
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
run: archery docker run ubuntu-docs
- name: Docker Push
if: success() && github.event_name == 'push' && github.repository == 'apache/arrow'
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
continue-on-error: true
run: archery docker push ubuntu-docs
10 changes: 7 additions & 3 deletions .github/workflows/docs_light.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,12 @@ on:
- 'ci/scripts/cpp_build.sh'
- 'ci/scripts/python_build.sh'

permissions:
contents: read

env:
ARROW_ENABLE_TIMING_TESTS: OFF
DOCKER_VOLUME_PREFIX: ".docker/"
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}

jobs:

Expand All @@ -49,7 +50,7 @@ jobs:
with:
fetch-depth: 0
- name: Cache Docker Volumes
uses: actions/cache@v2
uses: actions/cache@v3
with:
path: .docker
key: conda-docs-${{ hashFiles('cpp/**') }}
Expand All @@ -61,4 +62,7 @@ jobs:
- name: Setup Archery
run: pip install -e dev/archery[docker]
- name: Execute Docker Build
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
run: archery docker run conda-python-docs
23 changes: 20 additions & 3 deletions .github/workflows/go.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,8 @@ concurrency:
group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
cancel-in-progress: true

env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
permissions:
contents: read

jobs:

Expand Down Expand Up @@ -72,9 +71,15 @@ jobs:
- name: Setup Archery
run: pip install -e dev/archery[docker]
- name: Execute Docker Build
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
run: archery docker run debian-go
- name: Docker Push
if: success() && github.event_name == 'push' && github.repository == 'apache/arrow'
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
continue-on-error: true
run: archery docker push debian-go

Expand Down Expand Up @@ -108,9 +113,15 @@ jobs:
- name: Setup Archery
run: pip install -e dev/archery[docker]
- name: Execute Docker Build
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
run: archery docker run debian-go-cgo
- name: Docker Push
if: success() && github.event_name == 'push' && github.repository == 'apache/arrow'
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
continue-on-error: true
run: archery docker push debian-go-cgo

Expand Down Expand Up @@ -144,9 +155,15 @@ jobs:
- name: Setup Archery
run: pip install -e dev/archery[docker]
- name: Execute Docker Build
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
run: archery docker run debian-go-cgo-python
- name: Docker Push
if: success() && github.event_name == 'push' && github.repository == 'apache/arrow'
env:
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
continue-on-error: true
run: archery docker push debian-go-cgo-python

Expand Down
Loading

0 comments on commit 32f2665

Please sign in to comment.