Pipelines v0.20+ release broken for Kubernetes v1.16.x

[wlynch]

Expected Behavior

Tekton installation docs say Pipelines is compatible with Kubernetes >= v1.16

Actual Behavior

Attempting to upgrade from Pipelines v0.19.0 to v0.21 (and v0.20) results in the following error:

Error from server (Invalid): error when creating "https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.20.0/release.yaml": ValidatingWebhookConfiguration.admissionregistration.k8s.io "validation.webhook.pipeline.tekton.dev" is invalid: webhooks[0].admissionReviewVersions: Invalid value: []string{"v1"}: must include at least one of v1beta1
Error from server (Invalid): error when creating "https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.20.0/release.yaml": MutatingWebhookConfiguration.admissionregistration.k8s.io "webhook.pipeline.tekton.dev" is invalid: webhooks[0].admissionReviewVersions: Invalid value: []string{"v1"}: must include at least one of v1beta1
Error from server (Invalid): error when creating "https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.20.0/release.yaml": ValidatingWebhookConfiguration.admissionregistration.k8s.io "config.webhook.pipeline.tekton.dev" is invalid: webhooks[0].admissionReviewVersions: Invalid value: []string{"v1"}: must include at least one of v1beta1

Additionally, the pipeline and webhook controllers are in CrashLookBackOff with this error in the logs:

{"level":"fatal","ts":"2021-03-02T23:08:30.079Z","logger":"tekton-pipelines-webhook","caller":"sharedmain/main.go:287","msg":"Version check failed","commit":"49244c6","error":"pre-release kubernetes version \"1.16.15-gke.7800\" is not compatible, need at least \"1.17.0-\" (this can be overridden with the env var \"KUBERNETES_MIN_VERSION\"); note pre-release version is smaller than the corresponding release version (e.g. 1.x.y-z < 1.x.y), using 1.x.y-0 as the minimum version is likely to help in this case","stacktrace":"github.com/tektoncd/pipeline/vendor/knative.dev/pkg/injection/sharedmain.CheckK8sClientMinimumVersionOrDie\n\tgithub.com/tektoncd/pipeline/vendor/knative.dev/pkg/injection/sharedmain/main.go:287\ngithub.com/tektoncd/pipeline/vendor/knative.dev/pkg/injection/sharedmain.MainWithConfig\n\tgithub.com/tektoncd/pipeline/vendor/knative.dev/pkg/injection/sharedmain/main.go:184\nmain.main\n\tgithub.com/tektoncd/pipeline/cmd/webhook/main.go:235\nruntime.main\n\truntime/proc.go:204"}

Despite the error, tkn reports v0.21 after applying.

Steps to Reproduce the Problem

1. Start from Tekton Pipelines v0.19.0
2. kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.21.0/release.yaml

Additional Info

• Kubernetes version:

  Client Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.13-dispatcher", GitCommit:"fd22db44e150011eccc8729db223945384460143", GitTreeState:"clean", BuildDate:"2020-07-24T07:27:52Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
  Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.15-gke.7800", GitCommit:"cef3156c566a1d1a4b23ee360a760f45bfbaaac1", GitTreeState:"clean", BuildDate:"2020-12-14T09:12:37Z", GoVersion:"go1.13.15b4", Compiler:"gc", Platform:"linux/amd64"}
  

• Tekton Pipeline version:

  Client version: 0.15.0
  Pipeline version: v0.21.0
  

I've verified that v0.21.0 installation works as expected on a clean 1.18 cluster.

I think that #3640 introduced the admissionReviewVersions breakage, but the knative error implies we have deeper dependencies on 1.17 that might be harder to weed out.

It's been 2 releases and no one has appeared to notice until now(?), so it might make more sense to just rev the supported version in the docs and say that Tekton starting v0.20 requires v1.17. Not sure what the deprecation implications are here though.

[afrittoli]

Uhm, it looks like we forgot to update the installation guide. It was definitely in the release notes.

[jerop]

besides #3640, knative version was bumped to v0.20 in #3605 which bumped the min Kubernetes version to 1.17

the upgrade notice that Kubernetes 1.17+ is required to run Pipelines was included in v0.20.0, added it to v0.21.0 too

@wlynch is this the doc you're referring to or it's somewhere else as well?

[wlynch]

Uhm, it looks like we forgot to update the installation guide. It was definitely in the release notes.

the upgrade notice that Kubernetes 1.17+ is required to run Pipelines was included in v0.20.0, added it to v0.21.0 too

phew I think that takes care of the deprecation worries then.

@wlynch is this the doc you're referring to or it's somewhere else as well?

https://github.com/tektoncd/pipeline/blob/master/docs/install.md is where I noticed it - though there's a few other places I can see as well. I can throw together a PR to update the docs tomorrow!

/assign me

[tekton-robot]

@wlynch: GitHub didn't allow me to assign the following users: me.

Note that only tektoncd members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

Uhm, it looks like we forgot to update the installation guide. It was definitely in the release notes.

the upgrade notice that Kubernetes 1.17+ is required to run Pipelines was included in v0.20.0, added it to v0.21.0 too

phew I think that takes care of the deprecation issue then.

@wlynch is this the doc you're referring to or it's somewhere else as well?

https://github.com/tektoncd/pipeline/blob/master/docs/install.md is where I noticed it. I can throw together a PR to update the docs tomorrow!

/assign me

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[wlynch]

/assign @wlynch