Changes to triggering pipeline automation

[rossf7]

Hey @incertum @mickael-carl
we wanted to give you an update on what we're working on in WG Green Reviews post kubecon EU and get your feedback on the developments we have planned for the measurement pipeline.

We've identified 4 proposals (tracking issue #83) we're calling Deploy, Run, Report plus a 4th which is an investigation into Benchmarking. So far we've started speccing Deploy and Run.

1. Deploy - trigger GitHub Action workflow from an upstream release, including the binary
2. Run - run benchmark tests through GitHub Action

For the release automation we plan to use the Atom feed that GitHub publishes i.e. https://github.com/falcosecurity/falco/releases.atom

For testing the pipeline you'll be able to trigger it by sending a workflow dispatch event. We'll provide a GitHub access token for this.

Example curl command

curl -X POST \
     -H "Accept: application/vnd.github.v3+json" \
     -H "Authorization: token $GITHUB_PAT" \
     https://api.github.com/repos/cncf-tags/green-reviews-tooling/actions/workflows/pipeline.yaml/dispatches \
     -d '{"ref":"main", "inputs": {"cncf_project": "falco", "cncf_project_sub": "modern-ebpf","version":"0.37.0"}}'

Note: We may need to add more input params for the Run and Report steps.

The full proposal is being worked on in cncf-tags/green-reviews-tooling#88

Your feedback on the design is most welcome. Also please let us know if there are additional points when you would like to trigger the pipeline. Thanks! 🙏

cc @nikimanoledaki @AntonioDiTuri

[incertum]

Thanks a bunch @rossf7 for the updates 🚀 . At a high level and first glance this all sounds good to me and doable. I'll be watching as the approach gets shaped!

[rossf7]

@incertum thank you that's good to hear. We'll keep you updated on progress.

[rossf7]

Hi @incertum,
I wanted to update you on progress. The 1st proposal for Deploy is merged and the 2nd proposal for the Run step that @nikimanoledaki and Flavia are working on is in review.

cc @AntonioDiTuri

Related to the proposals there are 3 changes I'd like to propose to the structure here that I'd like to get your feedback on.

1. kustomization per driver

We'd like to have a separate kustomization for each driver to be able to deploy each independently.

We will deploy, benchmark and then uninstall each driver sequentially by setting concurrency in the workflow.

2. New node selector

Instead of having the driver name in the node selector we'd like to use a new node selector for all drivers.

node-role.kubernetes.io/benchmark: "true"

3. Moving benchmarks to GitHub Actions workflows

To prepare for the Run step we'd like to remove the benchmark manifests. As the proposal is they will be replaced by GitHub Actions workflows.

You can preview the changes in this branch

https://github.com/falcosecurity/cncf-green-review-testing/compare/main...rossf7:cncf-green-review-testing:feat/kustomization-per-driver?expand=1

If you're happy with these changes we can open a PR for them but I wanted to get your thoughts on them first.

Thank you and apologies this is a bit long! 🙏

[rossf7]

Quick update after syncing with Niki and Antonio on this. The benchmark manifests are now moved to a new dir called /benchmark-tests with the new node selector.

[incertum]

Hey @rossf7 LGTM, happy to approve the PR right away!

[rossf7]

Awesome 🌟 thank you @incertum! PR #18