Idea: TekDoc tool, similar to godoc

[dlorenc]

Desired state

godoc hosts documentation for go projects. It finds these projects (scrapes them? not sure!) and looks for docstrings in the required format and uses that to generate (and i guess maintain? it must poll for changes) documentation, for example [this is some godoc generated docs for a function in tekton pipelines](https://godoc.org/github.com/tektoncd/pipeline/pkg/apis/pipeline/v1alpha1#ApplyArrayReplacements, specifically this function.

What we would like is to be able to run an host a similar tool, maybe called TekDoc, that would show us for all the Pipelines and Tasks in tektoncd/catalog:

• The name of the Pipeline / Task
• Link to the source
• A description
• All parameters with descriptions + defaults
• All input/output resources with descriptions + defaults

More requirements

(note we dont need to meet all of these right away, we can start with something super simple and iterate!)

• Other people have their own catalogs too, e.g. https://github.com/garethr/snyk-tekton. We should make it so this tool could also display those docs, both allowing for our tool to point at multiple catalogs, and making it so that ppl can run the tool themselves
• One day we might have more types in the catalog, like PipelineResources and tekton triggers types so we can't just assume Pipelines and Tasks only.
• We are making changes to Pipelines and Tasks at the moment, so we need to make sure that it wont be too hard to change the tool to deal with those changes
• We should allow for the tool to display versions of the Pipelines and Tasks

Current state

1. At the moment all usage info is in READMEs that are submitted with the Pipelines and Tasks, e.g. this one https://github.com/tektoncd/catalog/tree/master/conftest#conftest
2. We have description fields for params but not for Pipelines and Tasks themselves, and not for the PipelineResources that they use

Possible plan of attack

I tried to break it up into a few milestones, but some of it could happen in parallel!

[ Milestone 1: add the necessary info to the types ]

1. Add description field to Pipeline type
2. Add description field to Task type
3. Add description field to input & output resources in Task (Add description field to Resources in Tasks pipeline#1389)
4. Add description field to resources in Pipeline
5. Update Tasks in the catalog to use these fields (might have to wait for a Pipelines release to merge the change, our versioning policy for the catalog vs. pipelines is a bit adhoc at the moment, + @vdemeester )

Suggestion: assume markdown format for contents of descriptions

[ Milestone 2: create service and/or library that scrapes catalog info]

For this bit it would probably be good to understand how godoc scrapes & polls!

1. The tool/service/library should be able to be given a github repo, find the Tasks and Pipelines
2. The tool/service/library should from those Tasks and Pipelines be able to extract their docs

The idea is that the webserver in milestone 3 will use this, but we can develop and test it without needing to have the whole webserver completed.

[ Milestone 3: create webserver ]

We can probably just pick any ol' webserver, but we'd want to make sure we can deploy and run it in a container. Would be great if the webserver we chose could automatically emit metrics to let us know if its up and running and if there are any errors.

1. The webserver should invoke the service, then display all the Tasks + Pipelines + their docs

Additional info

• about godoc
• Note that we need to keep to tekton's development principles, e.g. the tool itself should be implemented with tests and docs as we go

Original description

We could automatically generate interface documentation for the Tasks/Pipelines in this repository from the input and output parameters, and display this usage documentation nicely!

[bobcatfish]

oooo that sounds super sweet

[vdemeester]

that would be really good ! auto-generated on an url like https://catalog.tekton.dev 😍

[bobcatfish]

Been talking with @imjasonh and @wlynch and a great first step (ignore what I put in the description!) for this would be a command line tool that can generate some markdown from the existing description fields.

For example given these input params:

catalog/buildpacks/buildpacks-v3.yaml

Lines 6 to 21 in 1b3ceee

	inputs:
	params:
	- name: BUILDER_IMAGE
	description: The image on which builds will run (must include v3 lifecycle and compatible buildpacks).
	- name: USE_CRED_HELPERS
	description: Use Docker credential helpers for Google's GCR, Amazon's ECR, or Microsoft's ACR.
	default: 'false'
	- name: CACHE
	description: The name of the persistent app cache volume
	default: empty-dir
	- name: USER_ID
	description: The user ID of the builder image user
	default: "1000"
	- name: GROUP_ID
	description: The group ID of the builder image user
	default: "1000"

We would want to generate this markdown:

https://github.com/tektoncd/catalog/tree/master/buildpacks#parameters

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[vdemeester]

/remove-lifecycle rotten
/remove-lifecycle stale
/reopen

[tekton-robot]

@vdemeester: Reopened this issue.

In response to this:

/remove-lifecycle rotten
/remove-lifecycle stale
/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bobcatfish]

@vdemeester now that we've got the hub, do you think this tool still makes sense?

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[vdemeester]

/remove-lifecycle stale

[jromero]

I found this as I was looking for something like what @bobcatfish mentioned. AFAICT, Hub would simply display the README file for the task. Would it be possible to have the Hub provide the technicals (parameters, outputs, workspaces, etc)? The README could then be less structured data and instead be more helpful information like samples, community links, etc.

[vdemeester]

I found this as I was looking for something like what @bobcatfish mentioned. AFAICT, Hub would simply display the README file for the task. Would it be possible to have the Hub provide the technicals (parameters, outputs, workspaces, etc)? The README could then be less structured data and instead be more helpful information like samples, community links, etc.

@jromero right, that would be something to bring to the @tektoncd/hub-maintainers team, but I tend to agree yes. It might even be on the hub roadmap 😉

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.