Skip to content
This repository has been archived by the owner on Dec 5, 2024. It is now read-only.

[workflows] POC using Knative GitHub eventsource #928

Closed
wants to merge 1 commit into from

Conversation

lbernick
Copy link
Member

This commit adds a POC for using Knative GitHub eventsource to create a webhook on a repo declared in a new repo CRD, given a personal access token and webhook secret. The sink for the GitHub eventsource is the Workflows eventlistener.

The user is expected to create a GitRepository with references to the necessary credentials before creating the Workflow. The GitHub eventsource connection status is reflected on the GitRepository.

The GitHub eventsource validates payloads coming from GitHub, so this functionality is removed from the generated workflow trigger. An interceptor is added to filter to only the repo of interest, assuming GitHub event payloads.

This commit also adds substitution for parameters of the form $(repos.foo.url).

Benefits:

  • GitHub source takes care of setting up GitHub webhook so user does not have to.
  • We can add support for more eventsources by adding a new Knative eventsource.

Not yet implemented:

  • cross-namespace repo references. This will allow repos to be defined once and used in multiple workflows.

Challenges:

  • A webhook is associated with a repo, but the events a webhook is associated with can come from multiple workflows. It's not clear how to separate the repo definition from the events definition, so the repo configuration is created by the user, and the events part of its spec is later updated by the workflows controller. The GitHubSource must accept at least one event type, so we have to create it with one even if it's not initially specified in the GitRepository.
  • GitRepository is a very thin wrapper over the GitHub eventsource. Its main value add is abstracting away the Workflows EventListener.
  • It would be nice to add a plugin mechanism to support connecting to e.g. GitLab or connecting via a GitHub app. This approach relies on the workflows controller anticipating GitHub event payloads, and assumes the connection is done via a basic webhook with a personal access token.
  • The GitHub eventsource uses knative serving for autoscaling based on the number of events. Because events are infrequent, a pod is created each time an event is received and torn down afterwards, adding latency.
  • The GitHub eventsource doesn't preserve GitHub event headers, which are used for filtering event types and sending the right events to the right workflows.

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

  • Includes docs (if user facing)
  • Commit messages follow commit message best practices
  • Commit messages includes a project tag in the subject - e.g. [OCI], [hub], [results], [task-loops]

See the contribution guide
for more details.

@tekton-robot
Copy link

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@tekton-robot tekton-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 16, 2022
@tekton-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please ask for approval from lbernick after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tekton-robot tekton-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Nov 16, 2022
This commit adds a POC for using Knative GitHub eventsource
to create a webhook on a repo declared in a new repo CRD, given a personal access token
and webhook secret. The sink for the GitHub eventsource is the Workflows eventlistener.

The user is expected to create a GitRepository with references to the necessary credentials
before creating the Workflow. The GitHub eventsource connection status is reflected on the GitRepository.

The GitHub eventsource validates payloads coming from GitHub, so this functionality
is removed from the generated workflow trigger. An interceptor is added to filter to only
the repo of interest, assuming GitHub event payloads.

This commit also adds substitution for parameters of the form $(repos.foo.url).

Benefits:
- GitHub source takes care of setting up GitHub webhook so user does not have to.
- We can add support for more eventsources by adding a new Knative eventsource.

Not yet implemented:
- cross-namespace repo references. This will allow repos to be defined once and used in multiple workflows.

Challenges:
- A webhook is associated with a repo, but the events a webhook is associated with can come from multiple workflows.
It's not clear how to separate the repo definition from the events definition, so the repo configuration is created
by the user, and the events part of its spec is later updated by the workflows controller.
The GitHubSource must accept at least one event type, so we have to create it with one even if it's not initially
specified in the GitRepository.
- GitRepository is a very thin wrapper over the GitHub eventsource. Its main value add is abstracting away the
Workflows EventListener.
- It would be nice to add a plugin mechanism to support connecting to e.g. GitLab or connecting via a GitHub app.
This approach relies on the workflows controller anticipating GitHub event payloads, and assumes the connection
is done via a basic webhook with a personal access token.
- The GitHub eventsource uses knative serving for autoscaling based on the number of events. Because events are
infrequent, a pod is created each time an event is received and torn down afterwards, adding latency.
- The GitHub eventsource doesn't preserve GitHub event headers, which are used for filtering event types and
sending the right events to the right workflows.
@tekton-robot tekton-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 17, 2022
@lbernick lbernick closed this Jan 10, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants