Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor CI for forked repositories - POC #98

Merged
merged 45 commits into from
Mar 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
32e99a2
Refactor CI for forked repositories
yoanm Mar 14, 2024
d6bc4a2
Try
yoanm Mar 14, 2024
ea213e3
Fix?
yoanm Mar 14, 2024
a1ee6b2
Try
yoanm Mar 14, 2024
3a49eae
Enhance
yoanm Mar 14, 2024
22f5653
Try again
yoanm Mar 14, 2024
6a42e4b
Enhance
yoanm Mar 14, 2024
47bef1b
Try
yoanm Mar 14, 2024
7604fd7
Try
yoanm Mar 14, 2024
ec6c03f
Enhance
yoanm Mar 14, 2024
070bd57
Enhance
yoanm Mar 15, 2024
2ca3416
Enhance
yoanm Mar 15, 2024
9cd6619
Fix
yoanm Mar 15, 2024
b4990fc
Enhance
yoanm Mar 15, 2024
435be78
Enhance
yoanm Mar 15, 2024
77def2d
Move part to workflow_run
yoanm Mar 15, 2024
46064e6
Move another part to workflow_run
yoanm Mar 15, 2024
0fd6445
Move another part to workflow_run
yoanm Mar 15, 2024
26e9bc1
Enhance
yoanm Mar 15, 2024
bf9a149
Enhance
yoanm Mar 15, 2024
5e1f171
Enhance
yoanm Mar 15, 2024
f497f81
Enhance
yoanm Mar 15, 2024
67bff65
Fix
yoanm Mar 15, 2024
1563bf2
Enhance
yoanm Mar 15, 2024
1b33ebb
Fix
yoanm Mar 15, 2024
6303c71
Fix
yoanm Mar 16, 2024
bb45ab6
Enhance
yoanm Mar 16, 2024
a19bb02
Try matrix usage on outputs
yoanm Mar 16, 2024
d7568c5
Test
yoanm Mar 16, 2024
30ea5bf
Enhance
yoanm Mar 17, 2024
530f205
Enhance
yoanm Mar 17, 2024
714ac6a
Enhance
yoanm Mar 17, 2024
d851ead
Improve
yoanm Mar 19, 2024
97bae9a
Improve
yoanm Mar 19, 2024
278f804
Move to JS actions
yoanm Mar 19, 2024
8ec19b9
Fix
yoanm Mar 19, 2024
3e8f2cf
Improve
yoanm Mar 19, 2024
0fab93e
Clean
yoanm Mar 20, 2024
455c1a1
Re-add static and nightly checks
yoanm Mar 20, 2024
6a42239
Untrusted inputs
yoanm Mar 20, 2024
cc16032
Debug
yoanm Mar 20, 2024
9e4ff43
Fix
yoanm Mar 20, 2024
9234752
Fix/Enhance
yoanm Mar 20, 2024
a7605bb
Remove useless patch check
yoanm Mar 20, 2024
c201afc
Fix output
yoanm Mar 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/actions/reports-group/.editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[{*.yml, *.json}]
indent_style = space
indent_size = 2
Empty file.
13 changes: 13 additions & 0 deletions .github/actions/reports-group/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
PKG_LIST := node-sdk find load-metadata create codecov-uploader codacy-uploader
TARGETS := configure install build package lint test

define FOR_EACH_PKG
@for PKG in $(PKG_LIST); do \
echo ">>>>>> $(1) \"$$PKG\" Package >>>>>>"; \
$(MAKE) -C $$PKG $(1) || exit 1; \
echo "<<<<<<\n"; \
done
endef

$(TARGETS):
$(call FOR_EACH_PKG,$@)
19 changes: 19 additions & 0 deletions .github/actions/reports-group/codacy-uploader/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
.PHONY: install
install:
@echo "Nothing to install !"

.PHONY: build
build:
@echo "Nothing to build !"

.PHONY: package
package:
@echo "Nothing to pack !"

.PHONY: lint
lint:
@echo "Nothing to lint !"

.PHONY: test
test:
@echo "Nothing to test !"
96 changes: 96 additions & 0 deletions .github/actions/reports-group/codacy-uploader/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
name: TODO
description: TODO

inputs:
path:
description: A group directory or a glob pattern in order to find group directories and merge their metadata
required: true
project-token:
description: Codacy project token for the repository
required: true
follow-symbolic-links:
description: |
Indicates whether to follow symbolic links when resolving `path`
default: 'true'

outputs:
groups:
description: TODO
value: ${{ steps.build-outputs.outputs.groups }}
reports:
description: TODO
value: ${{ steps.build-outputs.outputs.reports }}

runs:
using: "composite"
steps:
# Even if an input is marked as "required", an empty value (or no value) may be passed !
- name: Validate inputs
uses: actions/github-script@v7
env:
INPUT_MAP: ${{ toJson(inputs) }}
with:
script: |
return core.group(
'Validate inputs',
async () => Object.entries(JSON.parse(process.env.INPUT_MAP))
.forEach(([key, val]) => {
if (!val.trim()) { throw new Error(`Input required and not supplied: ${name}`); }
})
);

# @TODO move reports-group/load-metadata action to a dedicated repo and remove the checkout
- uses: actions/checkout@v4
with:
path: custom-action-repo

- name: Load groups metadata
id: load-metadata
uses: ./custom-action-repo/.github/actions/reports-group/load-metadata
with:
path: ${{ inputs.path }}
format: string # String in order to concatenate interesting values
glue-string: ',' # Ensure glue string as it's the expected one by the uploader
follow-symbolic-links: ${{ inputs.follow-symbolic-links }}

- name: Build uploader option
id: build-uploader-options
uses: actions/github-script@v7
env:
METADATA: ${{ steps.load-metadata.outputs.metadata }}
with:
script: |
core.info('Build uploader options');
const {METADATA} = process.env;

const metadata = JSON.parse(METADATA);
core.setOutput('coverage-reports', metadata.reportPaths); // Trusted path as it comes from trusted metadata (=from `reports-group/load-metadata`)

- name: Ensure at least one report to upload
if: ${{ '' == steps.build-uploader-options.outputs.coverage-reports }}
uses: actions/github-script@v7
with:
script: |
core.setFailed('Unable to retrieve any report to upload. Something wrong most likely happened !');

- name: Upload to codacy
id: upload
uses: codacy/codacy-coverage-reporter-action@v1
with:
coverage-reports: ${{ steps.build-uploader-options.outputs.coverage-reports }}
project-token: ${{ inputs.project-token }}

- name: Build action outputs
id: build-outputs
uses: actions/github-script@v7
env:
METADATA: ${{ steps.load-metadata.outputs.metadata }}
REPORTS: ${{ steps.build-uploader-options.outputs.coverage-reports }}
with:
script: |
core.info('Build output');
const {REPORTS, METADATA} = process.env;

const metadata = JSON.parse(METADATA);
core.setOutput('groups', metadata.path.split(',').join('\n')); // Trusted path as it comes from trusted metadata (=from `reports-group/load-metadata`)
core.setOutput('reports', REPORTS.split(',').join('\n')); // Trusted path, see `build-uploader-options` step
19 changes: 19 additions & 0 deletions .github/actions/reports-group/codecov-uploader/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
.PHONY: install
install:
@echo "Nothing to install !"

.PHONY: build
build:
@echo "Nothing to build !"

.PHONY: package
package:
@echo "Nothing to pack !"

.PHONY: lint
lint:
@echo "Nothing to lint !"

.PHONY: test
test:
@echo "Nothing to test !"
117 changes: 117 additions & 0 deletions .github/actions/reports-group/codecov-uploader/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
name: TODO
description: TODO

inputs:
path:
description: A group directory or a glob pattern in order to find group directories and merge their metadata
required: true
token:
description: Codecov upload token
required: true
follow-symbolic-links:
description: |
Indicates whether to follow symbolic links when resolving `path`
default: 'true'

outputs:
groups:
description: TODO
value: ${{ steps.build-outputs.outputs.groups }}
name:
description: TODO
value: ${{ steps.build-outputs.outputs.name }}
reports:
description: TODO
value: ${{ steps.build-outputs.outputs.reports }}
flags:
description: TODO
value: ${{ steps.build-outputs.outputs.flags }}

runs:
using: "composite"
steps:
# Even if an input is marked as "required", an empty value (or no value) may be passed !
- name: Validate inputs
uses: actions/github-script@v7
env:
INPUT_MAP: ${{ toJson(inputs) }}
with:
script: |
return core.group(
'Validate inputs',
async () => Object.entries(JSON.parse(process.env.INPUT_MAP))
.forEach(([key, val]) => {
if (!val.trim()) { throw new Error(`Input required and not supplied: ${name}`); }
})
);

# @TODO move reports-group/load-metadata action to a dedicated repo and remove the checkout
- uses: actions/checkout@v4
with:
path: custom-action-repo

- name: Load groups metadata
id: load-metadata
uses: ./custom-action-repo/.github/actions/reports-group/load-metadata
with:
path: ${{ inputs.path }}
format: string # String in order to concatenate interesting values
glue-string: ',' # Ensure glue string as it's the expected one by the uploader
follow-symbolic-links: ${{ inputs.follow-symbolic-links }}

- name: Build uploader option
id: build-uploader-options
uses: actions/github-script@v7
env:
METADATA: ${{ steps.load-metadata.outputs.metadata }}
with:
script: |
core.info('Build uploader options');
const {METADATA} = process.env;

const metadata = JSON.parse(METADATA);
core.setOutput('name', metadata.name);
core.setOutput('files', metadata.reportPaths); // Trusted path as it comes from trusted metadata (=from `reports-group/load-metadata`)
if (metadata.flags.length > 0) {
core.setOutput('flags', metadata.flags);
}

- name: Ensure at least one report to upload
if: ${{ '' == steps.build-uploader-options.outputs.files }}
uses: actions/github-script@v7
with:
script: |
core.setFailed('Unable to retrieve any report to upload. Something wrong most likely happened !');

- name: Upload to codacy
id: upload
uses: codecov/codecov-action@v4
with:
token: ${{ inputs.token }}
name: ${{ steps.build-uploader-options.outputs.name }}
files: ${{ steps.build-uploader-options.outputs.files }}
flags: ${{ steps.build-uploader-options.outputs.flags }}
# underlying CLI behavior
disable_search: true
# GHAction behavior
fail_ci_if_error: true
verbose: ${{ runner.debug == '1' }}

- name: Build action outputs
id: build-outputs
uses: actions/github-script@v7
env:
METADATA: ${{ steps.load-metadata.outputs.metadata }}
REPORT_NAME: ${{ steps.build-uploader-options.outputs.name }}
REPORT_FILES: ${{ steps.build-uploader-options.outputs.files }}
REPORT_FLAGS: ${{ steps.build-uploader-options.outputs.flags }}
with:
script: |
core.info('Build output');
const {REPORT_NAME, REPORT_FILES, REPORT_FLAGS, METADATA} = process.env;

const metadata = JSON.parse(METADATA);
core.setOutput('group', metadata.path.split(',').join('\n')); // Trusted path as it comes from trusted metadata (=from `reports-group/load-metadata`)
core.setOutput('name', REPORT_NAME);
core.setOutput('reports', REPORT_FILES.split(',').join('\n')); // Trusted path, see `build-uploader-options` step
core.setOutput('flags', undefined !== REPORT_FLAGS ? REPORT_FLAGS.split(',').join('\n') : '');
1 change: 1 addition & 0 deletions .github/actions/reports-group/create/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
dist/** -diff linguist-generated=true
2 changes: 2 additions & 0 deletions .github/actions/reports-group/create/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.yarn
node_modules
1 change: 1 addition & 0 deletions .github/actions/reports-group/create/.nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
20
22 changes: 22 additions & 0 deletions .github/actions/reports-group/create/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
.PHONY: install
install:
yarn install

.PHONY: build
build: install package

.PHONY: package
package:
rm -Rf dist/* && ./node_modules/.bin/ncc build index.js --minify --source-map --license LICENSE --out dist

.PHONY: lint
lint:
./node_modules/.bin/eslint index.js

.PHONY: test
test:
echo "Error: no test specified"
exit 1

get-action-nodejs-version: ## Display node version configured on action.yml
@grep -E "using:\s*'?node" action.yml | sed -e "s/^.*using: '*node\([0-9][0-9]\)'*.*$$/\1/"
37 changes: 37 additions & 0 deletions .github/actions/reports-group/create/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: create
description: |
Create a reports group directory under provided `path`, with a metada file and a copy of all `files`
Reports are expected to share the same format !
inputs:
name:
description: Group name
required: true
format:
description: "`clover`, `cobertura`, `lcov`, etc"
required: true
files:
description: Multiline list of reports attached to the group
required: true
flags:
description: Multiline list of flags attached to the group
default: ''
path:
description: Root directory for the group directory (default to current directory)
default: '.'
follow-symbolic-links:
description: |
Indicates whether to follow symbolic links when resolving `files`
default: 'true'

outputs:
path:
description: Path to the group directory
reports:
description: Multiline List of created reports for the group
files:
description: Multiline list of reports attached to the group.
May be different than provided `files` in case of glob pattern !

runs:
using: 'node20'
main: 'dist/index.js'
Empty file.
Loading
Loading