Exclude manifests for deprecated object versions

[maxdanilov]

Hi there,

We're using kubeval to validate manifests, as well as to catch deprecation warnings (before upgrading to the new version of k8s).

Unfortunately, as far as I can see, this repository includes deprecated manifests alongside with normal ones, and fires a false negative in case an object is passing a validation but the schema is marked as deprecated.

Example:

when running kubeval manifest.yml -v1.16.0 I would expect the following manifest to fail the kubeval checking:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: depl
  namespace: default

since extensions.v1beta1.Deployment is not supported in k8s 1.16.0 anymore (it's been moved to apps.v1.Deployment)

However, it passes, since the manifests for all old API groups for Deployment are still present in the repo, e.g.:

• https://github.com/instrumenta/kubernetes-json-schema/blob/master/v1.16.0/deployment-apps-v1beta1.json
• https://github.com/instrumenta/kubernetes-json-schema/blob/master/v1.16.0/deployment-apps-v1beta2.json
• https://github.com/instrumenta/kubernetes-json-schema/blob/master/v1.16.0/deployment-extensions-v1beta1.json

Not sure how this can be solved (as it seems coming from actual manifests for 1.16.0 that do have the manifests present), but one obvious solution could be creating a new set of manifest groups (e.g. v1.16.0-no-deprecated, v1.17.0-no-deprecated etc.) with manifests that have a DEPRECATED string in their description excluded from it:
https://github.com/instrumenta/kubernetes-json-schema/blob/master/v1.16.0/deployment-extensions-v1beta1.json#L2

[maxdanilov]

Similar issue in kubeval repo: instrumenta/kubeval#189

[maxdanilov]

@garethr do you have an opinion on this one?

[cmur2]

I would be interested in some way to tell deprecated manifests apart from normal ones for a certain Kubernetes version - it would be great to have some input here :)

[nicorikken]

I came here for the exact same use-case. Have been suggesting some ideas in the kubeval repo. Having an explicit keyword the schema would help. Rather than having to look for the word 'deprecated' in the description string. And the other solution is to have separate folders. But I think deprecation is orthogonal to the type of schema (local, standalone, strict). Having a keyword would prevent doubling the amount of directories for with/without deprecated schema's.

Also, not all deprecations have full-caps DEPRECATED in the description. Consider this extensions/v1beta1/podSecurityPolicy

[maxdanilov]

@nicorikken
We gave up on the idea of using kubeval for this, instead for deprecation checks we run https://github.com/open-policy-agent/conftest using REGO policies (https://github.com/swade1987/deprek8ion/tree/master/policies could be a source of inspiration for the policies)