Proposal: Custom task SDK.

[ScrapCodes]

Proposal for creating tektoncd/experimental/custom-task-sdk to incubate Custom Task SDK project

Per process, this project was proposed with verbal support at the Tekton API Working Group on June 14th 9am PST

The problem the project will solve

A custom task SDK will make the work of Custom Task author easier and will offer standard way to implement custom task controllers. While adhering to current separation of concern between a custom task controller's role and tektoncd owned controller's role, it should make custom tasks easier to manage and reason by tektoncd.

Some highlights:

1. A custom task SDK may exist as a template project on github. A custom task developer will be able to fill in minimal details and focus on the actual controller logic than building the whole thing from scratch. It should have sufficient documentation and in code comments to give guidance to the developer, on the idiomatic ways.
2. Provide helper functions for more advanced use cases.
3. A custom task may be able to profess what features it would like to support. For example, does it support cancel/timeout or retries? Or does it support taskSpec or not.
4. Since a custom task definition can be embedded inside a PipelineRun yaml, the template sdk will include a custom task specific cli tool for validating embedded specs.
5. SDK can include stub code for validation, cancelation and even retrying to the end users, with documentation for the recommended implementations. For example, we recommend async validation, rather than admission webhook based.

Related TEPs:

1. TEP-0002
2. TEP-0061

Related issues:

1. Implement initial update timeout for custom tasks. pipeline#3997

Who will own it

• @ScrapCodes

Updates

TBD

---

/cc @vdemeester @afrittoli @pritidesai @jerop @nikhil-thomas @bobcatfish

[pritidesai]

We are tracking approval from governance with this issue, please vote for the approval.

[pritidesai]

Reference: #304

Need reviewers:

@vdemeester
@afrittoli
@pritidesai
@jerop
@nikhil-thomas

[waveywaves]

1. A custom task SDK may exist as a template project on github. A custom task developer will be able to fill in minimal details and focus on the actual controller logic than building the whole thing from scratch. It should have sufficient documentation and in code comments to give guidance to the developer, on the idiomatic ways.

+1 for this project propsal seeing it as parallel to what is there in Jenkins with the plugin development environment. Creating custom task controllers by defining the custom task layout and basic logic of the controller and the developer can focus on the reconciliation of the custom task and it's lifecycle is a great idea !
There is a project template I had created a few days ago for partially the same purpose.

3. Since a custom task definition can be embedded inside a PipelineRun yaml, the template sdk will include a custom task specific cli tool for validating embedded specs.
   Why not make it a part of the tkn cli ? Are you also thinking of allowing users to initialize Custom Tasks based on exiting archetypes something like this ?

Considering Custom Task Reconciliation will run in it's own controller away from the main Tekton controllers, it shouldn't have sideeffects on the Tekton controllers (and should probably be taken care of by the SDK). This is something that needs to be kept in mind, otherwise "plugin hell" which is a huge problem in Jenkins might get reflected in due time.

4. Since a custom task definition can be embedded inside a PipelineRun yaml, the template sdk will include a custom task specific cli tool for validating embedded specs.

✨

Maybe we can call this sdk the Tekton Plugin SDK 😉 instead of Custom Task SDK ? +1 for Custom Task SDK from my side, but was thinking if using "Plugin" might take away the few minutes of explantaion it might take us to explain what a Custom Task is to existing CI/CD users/developers. What a Custom Task is would have to be understood by the developer at some point but using the work "Plugin" might (a very strong might) hockey-stick developer involvement in creating new CustomTask controllers (or tekton plugins 😉).

[ScrapCodes]

Thanks @waveywaves :)

Why not make it a part of the tkn cli ?

It makes more sense to do that. However, it is not yet clear how it will work. Will it be like tkn custom-task validate -f ... (similar to kubectl plugin) ? Or something else. It needs approvals from CLI team etc.. So skipped the exact details. While designing we can delve deeper.

At the moment, custom task seems to be ubiquitous in tekton world. Changing this name means we would have to update the current documentation/code base and so on. To me Plugin means a lot more than what custom task can do today.

[waveywaves]

It makes more sense to do that. However, it is not yet clear how it will work. Will it be like tkn custom-task validate -f ... (similar to kubectl plugin) ? Or something else. It needs approvals from CLI team etc.. So skipped the exact details. While designing we can delve deeper.

I agree, the cli could be started in the experimental repo to begin with and then the cobra module can be moved to tkn in the near future.

At the moment, custom task seems to be ubiquitous in tekton world. Changing this name means we would have to update the current documentation/code base and so on. To me Plugin means a lot more than what custom task can do today.

We keep using Custom Task, just proposing if the SDK be called Tekton Plugin SDK. Not saying that we should change the name of the Tekton resource from Custom Task to Plugin.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[vdemeester]

/reopen

[tekton-robot]

@vdemeester: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.