Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflow refactor #5766

Merged
merged 48 commits into from
Jun 19, 2024
Merged
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
891858f
refactor main CI workflow
pdabelf5 Mar 22, 2024
5c1de47
testing azure upload
pdabelf5 Jun 7, 2024
5415dcb
added dry_run to azure upload
pdabelf5 Jun 12, 2024
e400a46
remove excess permissions
pdabelf5 Jun 12, 2024
f35b132
add github release to release workflow
pdabelf5 Jun 12, 2024
7c95c94
notify if release pipeline fails
pdabelf5 Jun 12, 2024
c95231f
add slack notifications
pdabelf5 Jun 13, 2024
ea2d395
Fix test_dos_sec_logs_on (#5725)
pasmant Jun 12, 2024
bd93b6b
enable external service args (#5736)
vepatel Jun 12, 2024
f6717e1
Integrate NAP WAF v5 (#5698)
shaun-nx Jun 12, 2024
580940a
Bump github/codeql-action from 3.25.8 to 3.25.9 in the actions group …
dependabot[bot] Jun 13, 2024
bf146cd
Mount config and deployments dirs to smoke containers (#5740)
oseoin Jun 13, 2024
f4d4168
Add F5 prefix to configuration opening paragraphs, style consistency …
ADubhlaoich Jun 13, 2024
2028376
Docker image update d41d8cd9 (#5743)
nginx-bot Jun 13, 2024
9f37c5f
add release workflow (#5742)
pdabelf5 Jun 14, 2024
d7cec6a
Merge branch 'main' into workflow-refactor
pdabelf5 Jun 17, 2024
5ab9425
correct wafv5 release
pdabelf5 Jun 17, 2024
dda4ac8
renamed inputs on oss/plus build workflows
pdabelf5 Jun 17, 2024
c0db353
correct workflow permissions
pdabelf5 Jun 17, 2024
ce236d8
additional permissions for workflow
pdabelf5 Jun 17, 2024
6e9a640
remove duplicate md5sum
pdabelf5 Jun 18, 2024
fe66b7d
remove unused cosign & syft downloads
pdabelf5 Jun 18, 2024
59e0e62
Merge branch 'main' into workflow-refactor
oseoin Jun 18, 2024
5c45b0b
fix image exists conditionals in ci
oseoin Jun 18, 2024
86b0264
add debug output
oseoin Jun 18, 2024
425193a
Merge branch 'workflow-refactor' of github.com:nginxinc/kubernetes-in…
pdabelf5 Jun 18, 2024
4e16ff5
ensure base image variable is set
pdabelf5 Jun 18, 2024
e36aa3e
simplify workflow for debugging
pdabelf5 Jun 18, 2024
f8aa05d
switch input to boolean
pdabelf5 Jun 18, 2024
18e93b2
ensure full-build is always a bool
pdabelf5 Jun 18, 2024
039c4fc
uncomment disabled jobs
pdabelf5 Jun 18, 2024
39bf7c1
fully validate source branch string
pdabelf5 Jun 18, 2024
d77bb65
ensure boolean logic for "force" variable in image promotion workflow
pdabelf5 Jun 18, 2024
a422598
fix setcap prebuild not working
oseoin Jun 18, 2024
695200d
Merge branch 'workflow-refactor' of github.com:nginxinc/kubernetes-in…
oseoin Jun 18, 2024
be35a57
Merge branch 'workflow-refactor' of github.com:nginxinc/kubernetes-in…
pdabelf5 Jun 18, 2024
785a893
remove PR tags
oseoin Jun 18, 2024
7273936
fix image build logic
oseoin Jun 18, 2024
bc9ffa7
remove skopeo sha
oseoin Jun 18, 2024
443b4be
Merge branch 'main' into workflow-refactor
oseoin Jun 18, 2024
19f28d5
change how IC_VERSION is calculated, fix version label bug
oseoin Jun 18, 2024
af932a7
add cache miss failures for oss and plus
oseoin Jun 18, 2024
d760391
Merge branch 'main' into workflow-refactor
oseoin Jun 18, 2024
fb806bb
Merge branch 'workflow-refactor' of github.com:nginxinc/kubernetes-in…
pdabelf5 Jun 18, 2024
1ed6f82
Merge branch 'main' into workflow-refactor
oseoin Jun 18, 2024
93f32bb
add wafv5 to edge publish flow
pdabelf5 Jun 18, 2024
e9c11cd
Merge branch 'workflow-refactor' of github.com:nginxinc/kubernetes-in…
pdabelf5 Jun 18, 2024
47fb333
correct waf v5 retag
pdabelf5 Jun 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/scripts/variables.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ get_chart_md5() {
}

get_actions_md5() {
find .github .github/data/version.txt -type f -exec md5sum {} + | LC_ALL=C sort | md5sum | awk '{ print $1 }'
find .github -type f -exec md5sum {} + | LC_ALL=C sort | md5sum | awk '{ print $1 }'
}

get_build_tag() {
Expand Down
207 changes: 74 additions & 133 deletions .github/workflows/build-oss.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,21 +9,29 @@ on:
image:
required: true
type: string
tag:
required: false
type: string
go-md5:
required: true
type: string
base-image-md5:
required: false
required: true
type: string
tag:
required: false
branch:
required: true
type: string
publish-image:
required: false
authenticated:
required: true
type: boolean
forked-workflow:
full-build:
description: Always build base image
required: false
type: boolean
default: false
ic-version:
required: false
type: string

defaults:
run:
Expand All @@ -38,85 +46,60 @@ jobs:
permissions:
contents: read # for docker/build-push-action to read repo content
security-events: write # for github/codeql-action/upload-sarif to upload SARIF results
id-token: write # for OIDC login to AWS ECR
id-token: write # for OIDC login to GCR
packages: write # for docker/build-push-action to push to GHCR
outputs:
version: ${{ steps.meta.outputs.version }}
image_digest: ${{ steps.build-push.outputs.digest }}
steps:
- name: Checkout Repository
uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
with:
ref: ${{ inputs.tag != '' && format('refs/tags/v{0}', inputs.tag) || github.ref }}
ref: ${{ inputs.branch }}
fetch-depth: 0

- name: Fetch Cached Artifacts
uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
with:
path: ${{ github.workspace }}/dist
key: nginx-ingress-${{ inputs.go-md5 }}

- name: Setup QEMU
uses: docker/setup-qemu-action@68827325e0b33c7199eb31dd4e31fbe9023e06e3 # v3.0.0
with:
platforms: arm,arm64,ppc64le,s390x
if: ${{ inputs.publish-image }}

- name: Docker Buildx
uses: docker/setup-buildx-action@d70bba72b1f3fd22344832f00baa16ece964efeb # v3.3.0

- name: DockerHub Login
uses: docker/login-action@0d4c9c5ea7693da7b068278f7b52bda2a190a446 # v3.2.0
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
if: ${{ inputs.publish-image }}

- name: Login to GitHub Container Registry
uses: docker/login-action@0d4c9c5ea7693da7b068278f7b52bda2a190a446 # v3.2.0
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
if: ${{ inputs.publish-image }}

- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
aws-region: us-east-1
role-to-assume: ${{ secrets.AWS_ROLE_PUBLIC_ECR }}
if: ${{ inputs.publish-image }}

- name: Login to Public ECR
uses: docker/login-action@0d4c9c5ea7693da7b068278f7b52bda2a190a446 # v3.2.0
with:
registry: public.ecr.aws
if: ${{ inputs.publish-image }}

- name: Login to Quay.io
uses: docker/login-action@0d4c9c5ea7693da7b068278f7b52bda2a190a446 # v3.2.0
with:
registry: quay.io
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_ROBOT_TOKEN }}
if: ${{ inputs.publish-image }}

- name: Authenticate to Google Cloud
id: auth
uses: google-github-actions/auth@71fee32a0bb7e97b4d33d548e7d957010649d8fa # v2.1.3
with:
token_format: access_token
workload_identity_provider: ${{ secrets.GCR_WORKLOAD_IDENTITY }}
service_account: ${{ secrets.GCR_SERVICE_ACCOUNT }}
if: ${{ ! inputs.forked-workflow }}
if: ${{ inputs.authenticated }}

- name: Login to GCR
uses: docker/login-action@0d4c9c5ea7693da7b068278f7b52bda2a190a446 # v3.2.0
with:
registry: gcr.io
username: oauth2accesstoken
password: ${{ steps.auth.outputs.access_token }}
if: ${{ ! inputs.forked-workflow }}
if: ${{ inputs.authenticated }}

- name: Docker meta
id: meta
uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81 # v5.5.1
with:
context: workflow
images: |
name=gcr.io/f5-gcs-7899-ptg-ingrss-ctlr/dev/nginx-ic/nginx-ingress
flavor: |
suffix=${{ contains(inputs.image, 'ubi') && '-ubi' || '' }}${{ contains(inputs.image, 'alpine') && '-alpine' || '' }}
tags: |
type=raw,value=${{ inputs.tag }}
labels: |
org.opencontainers.image.description=NGINX Ingress Controller for Kubernetes
io.artifacthub.package.readme-url=https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/main/README.md
io.artifacthub.package.logo-url=https://docs.nginx.com/nginx-ingress-controller/images/icons/NGINX-Ingress-Controller-product-icon.svg
io.artifacthub.package.maintainers=[{"name":"NGINX Inc","email":"[email protected]"}]
io.artifacthub.package.license=Apache-2.0
io.artifacthub.package.keywords=kubernetes,ingress,nginx,controller
env:
DOCKER_METADATA_ANNOTATIONS_LEVELS: manifest,index

- name: Check if base images exist
id: base_exists
Expand All @@ -126,64 +109,48 @@ jobs:
if docker manifest inspect ${base_image}; then
echo "exists=true" >> $GITHUB_OUTPUT
fi
if: ${{ ! inputs.forked-workflow }}
if: ${{ inputs.authenticated && ! inputs.full-build }}

- name: Build Base Container
uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5.4.0
with:
file: build/Dockerfile
context: "."
cache-from: type=gha,scope=${{ inputs.image }}
cache-to: type=gha,scope=${{ inputs.image }},mode=max
target: common
tags: ${{ steps.base_exists.outputs.image }}
platforms: ${{ inputs.platforms }}
pull: true
push: true
no-cache: true
build-args: |
BUILD_OS=${{ inputs.image }}
IC_VERSION=${{ inputs.tag }}
if: ${{ ! inputs.forked-workflow && steps.base_exists.outputs.exists != 'true' }}
IC_VERSION=${{ inputs.ic-version && inputs.ic-version || steps.meta.outputs.version }}
if: ${{ inputs.authenticated && steps.base_exists.outputs.exists != 'true' }}

- name: Check if target image exists
id: target_exists
run: |
if docker pull ${{ steps.meta.outputs.tags }}; then
echo "exists=true" >> $GITHUB_OUTPUT
fi
if: ${{ inputs.authenticated && ! inputs.full-build }}

- name: Get short tag
id: tag
- name: Debug values
run: |
version="${{ inputs.tag }}"
short="${version%.*}"
echo "short=$short" >> $GITHUB_OUTPUT
if: ${{ inputs.tag != '' }}
echo "authenticated: ${{ inputs.authenticated }}"
echo "base_exists: ${{ steps.base_exists.outputs.exists }}"
echo "target_exists: ${{ steps.target_exists.outputs.exists }}"
echo "full-build: ${{ inputs.full-build }}"
echo "all: ${{ inputs.authenticated || steps.base_exists.outputs.exists != 'true' || steps.target_exists.outputs.exists != 'true' }}"

- name: Docker meta
id: meta
uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81 # v5.5.1
- name: Fetch Cached Artifacts
uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
with:
context: ${{ inputs.tag != '' && 'git' || 'workflow' }}
images: |
name=nginx/nginx-ingress
name=ghcr.io/nginxinc/kubernetes-ingress
name=public.ecr.aws/nginx/nginx-ingress
name=quay.io/nginx/nginx-ingress
flavor: |
latest=${{ (inputs.tag != '' && 'true') || 'auto' }}
suffix=${{ contains(inputs.image, 'ubi') && '-ubi' || '' }}${{ contains(inputs.image, 'alpine') && '-alpine' || '' }},onlatest=true
tags: |
type=edge
type=ref,event=pr
type=ref,event=branch,enable=${{ startsWith(github.ref, 'refs/heads/release-') }}
type=schedule,enable=${{ inputs.tag == '' }}
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=${{ inputs.tag }},enable=${{ inputs.tag != '' }}
type=raw,value=${{ steps.tag.outputs.short }},enable=${{ inputs.tag != '' }}
labels: |
org.opencontainers.image.description=NGINX Ingress Controller for Kubernetes
io.artifacthub.package.readme-url=https://raw.githubusercontent.com/nginxinc/kubernetes-ingress/main/README.md
io.artifacthub.package.logo-url=https://docs.nginx.com/nginx-ingress-controller/images/icons/NGINX-Ingress-Controller-product-icon.svg
io.artifacthub.package.maintainers=[{"name":"NGINX Inc","email":"[email protected]"}]
io.artifacthub.package.license=Apache-2.0
io.artifacthub.package.keywords=kubernetes,ingress,nginx,controller
env:
DOCKER_METADATA_ANNOTATIONS_LEVELS: manifest,index
path: ${{ github.workspace }}/dist
key: nginx-ingress-${{ inputs.go-md5 }}
fail-on-cache-miss: true
if: ${{ inputs.authenticated || steps.base_exists.outputs.exists != 'true' || steps.target_exists.outputs.exists != 'true' }}

- name: Build Docker image
uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5.4.0
Expand All @@ -193,35 +160,21 @@ jobs:
context: "."
cache-from: type=gha,scope=${{ inputs.image }}
cache-to: type=gha,scope=${{ inputs.image }},mode=max
target: goreleaser${{ ! inputs.forked-workflow && '-prebuilt' || '' }}
target: goreleaser${{ inputs.authenticated && '-prebuilt' || '' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
annotations: ${{ github.event_name != 'pull_request' && steps.meta.outputs.annotations || '' }}
platforms: ${{ github.event_name != 'pull_request' && ! startsWith(github.ref, 'refs/heads/release-') && inputs.platforms || '' }}
load: ${{ github.event_name == 'pull_request' || startsWith(github.ref, 'refs/heads/release-') }}
push: ${{ inputs.publish-image }}
annotations: ${{ steps.meta.outputs.annotations }}
platforms: ${{ inputs.platforms }}
load: false
push: ${{ inputs.authenticated }}
pull: true
no-cache: ${{ inputs.publish-image }}
sbom: ${{ inputs.publish-image }}
sbom: ${{ inputs.authenticated }}
provenance: false
build-args: |
BUILD_OS=${{ inputs.image }}
${{ ! inputs.forked-workflow && format('PREBUILT_BASE_IMG={0}', steps.base_exists.outputs.image) || '' }}
IC_VERSION=${{ (github.event_name == 'pull_request' || startsWith(github.ref, 'refs/heads/release-')) && 'CI' || steps.meta.outputs.version }}

- name: Certify Images
continue-on-error: true
run: |
curl -fsSL https://github.com/redhat-openshift-ecosystem/openshift-preflight/releases/download/1.6.11/preflight-linux-amd64 --output preflight
chmod +x preflight

IFS=',' read -ra arch_list <<< "${{ inputs.platforms }}"

for arch in "${arch_list[@]}"; do
architecture=("${arch#*/}")
./preflight check container quay.io/nginx/nginx-ingress:${{ steps.meta.outputs.version }} --pyxis-api-token ${{ secrets.PYXIS_API_TOKEN }} --certification-project-id ${{ secrets.CERTIFICATION_PROJECT_ID }} --platform $architecture --submit
done
if: ${{ (github.ref_type == 'tag' && vars.OLD_RELEASE_FLOW == 'true') && contains(inputs.image, 'ubi') }}
${{ inputs.authenticated && format('PREBUILT_BASE_IMG={0}', steps.base_exists.outputs.image) }}
IC_VERSION=${{ inputs.ic-version && inputs.ic-version || steps.meta.outputs.version }}
if: ${{ steps.base_exists.outputs.exists != 'true' || steps.target_exists.outputs.exists != 'true' }}

- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@595be6a0f6560a0a8fc419ddf630567fc623531d # 0.22.0
Expand All @@ -231,12 +184,14 @@ jobs:
format: "sarif"
output: "trivy-results-${{ inputs.image }}.sarif"
ignore-unfixed: "true"
if: ${{ inputs.authenticated && steps.build-push.conclusion == 'success' }}

- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@530d4feaa9c62aaab2d250371e2061eb7a172363 # v3.25.9
continue-on-error: true
with:
sarif_file: "trivy-results-${{ inputs.image }}.sarif"
if: ${{ inputs.authenticated && steps.build-push.conclusion == 'success' }}

- name: Upload Scan Results
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4.3.3
Expand All @@ -245,17 +200,3 @@ jobs:
name: "trivy-results-${{ inputs.image }}.sarif"
path: "trivy-results-${{ inputs.image }}.sarif"
if: always()

send-notification:
name: Send Notification
needs: build
uses: ./.github/workflows/updates-notification.yml
with:
tag: ${{ inputs.tag }}
version: ${{ needs.build.outputs.version }}
image_digest: ${{ needs.build.outputs.image_digest }}
permissions:
contents: read
actions: read
secrets: inherit
if: ${{ inputs.tag != '' }}
Loading
Loading