[e2e] Check labels documentation

[olivierlemasle]

What this PR does / why we need it:
This PR adds a new e2e test, checking that all metrics and labels found in /metrics endpoint are correctly documented (currently, metrics documentation is already checked with make doccheck, but labels are not validated).

When part of the label name is documented in uppercase (e.g. label_DAEMONSET_LABEL), we will consider that by convention, it means that the uppercase part is dynamic (checked as a wildcard).

NB: we also remove test manifest for Lease; as explained in the commit message, this was not necessary and caused an (ignored) error at the end of e2e tests.

Which issue(s) this PR fixes:
Fixes #1078

[olivierlemasle]

CI failures are actually good news:

=== RUN   TestDocumentation
---- FAIL: TestDocumentation (0.08s)
-    main_test.go:131: Label horizontalpodautoscaler not found in documentation. Documented labels for metric kube_horizontalpodautoscaler_spec_min_replicas are: hpa, namespace
-    main_test.go:131: Label horizontalpodautoscaler not found in documentation. Documented labels for metric kube_horizontalpodautoscaler_spec_max_replicas are: hpa, namespace
-    main_test.go:131: Label horizontalpodautoscaler not found in documentation. Documented labels for metric kube_horizontalpodautoscaler_status_current_replicas are: hpa, namespace
-    main_test.go:112: Metric kube_validatingwebhookconfiguration_created not found in documentation.
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_allocatable are: node, resource
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_allocatable are: node, resource
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_allocatable are: node, resource
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_allocatable are: node, resource
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_allocatable are: node, resource
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_allocatable are: node, resource
-    main_test.go:112: Metric kube_pod_start_time not found in documentation.
-    main_test.go:112: Metric kube_mutatingwebhookconfiguration_created not found in documentation.
-    main_test.go:131: Label horizontalpodautoscaler not found in documentation. Documented labels for metric kube_horizontalpodautoscaler_status_condition are: hpa, namespace, condition, status
-    main_test.go:131: Label horizontalpodautoscaler not found in documentation. Documented labels for metric kube_horizontalpodautoscaler_status_condition are: hpa, namespace, condition, status
-    main_test.go:131: Label horizontalpodautoscaler not found in documentation. Documented labels for metric kube_horizontalpodautoscaler_status_condition are: hpa, namespace, condition, status
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_capacity are: node, resource
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_capacity are: node, resource
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_capacity are: node, resource
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_capacity are: node, resource
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_capacity are: node, resource
-    main_test.go:131: Label unit not found in documentation. Documented labels for metric kube_node_status_capacity are: node, resource
-    main_test.go:131: Label label_app_kubernetes_io_name not found in documentation. Documented labels for metric kube_replicaset_labels are: replicaset, namespace
-    main_test.go:131: Label label_app_kubernetes_io_version not found in documentation. Documented labels for metric kube_replicaset_labels are: replicaset, namespace
-    main_test.go:131: Label label_pod_template_hash not found in documentation. Documented labels for metric kube_replicaset_labels are: replicaset, namespace
-    main_test.go:131: Label label_k8s_app not found in documentation. Documented labels for metric kube_replicaset_labels are: replicaset, namespace
-    main_test.go:131: Label label_pod_template_hash not found in documentation. Documented labels for metric kube_replicaset_labels are: replicaset, namespace
-    main_test.go:131: Label horizontalpodautoscaler not found in documentation. Documented labels for metric kube_horizontalpodautoscaler_metadata_generation are: hpa, namespace
-    main_test.go:112: Metric kube_mutatingwebhookconfiguration_info not found in documentation.
-    main_test.go:112: Metric kube_validatingwebhookconfiguration_info not found in documentation.
-    main_test.go:131: Label label_k8s_app not found in documentation. Documented labels for metric kube_deployment_labels are: deployment, namespace
-    main_test.go:131: Label label_app_kubernetes_io_name not found in documentation. Documented labels for metric kube_deployment_labels are: deployment, namespace
-    main_test.go:131: Label label_app_kubernetes_io_version not found in documentation. Documented labels for metric kube_deployment_labels are: deployment, namespace
-    main_test.go:131: Label phase not found in documentation. Documented labels for metric kube_namespace_status_phase are: namespace, status
-    main_test.go:131: Label phase not found in documentation. Documented labels for metric kube_namespace_status_phase are: namespace, status
-    main_test.go:131: Label phase not found in documentation. Documented labels for metric kube_namespace_status_phase are: namespace, status
-    main_test.go:131: Label phase not found in documentation. Documented labels for metric kube_namespace_status_phase are: namespace, status
-    main_test.go:131: Label phase not found in documentation. Documented labels for metric kube_namespace_status_phase are: namespace, status
-    main_test.go:131: Label phase not found in documentation. Documented labels for metric kube_namespace_status_phase are: namespace, status
-    main_test.go:131: Label phase not found in documentation. Documented labels for metric kube_namespace_status_phase are: namespace, status
-    main_test.go:131: Label phase not found in documentation. Documented labels for metric kube_namespace_status_phase are: namespace, status
-    main_test.go:131: Label horizontalpodautoscaler not found in documentation. Documented labels for metric kube_horizontalpodautoscaler_spec_target_metric are: hpa, namespace, metric_name, metric_target_type
-    main_test.go:131: Label horizontalpodautoscaler not found in documentation. Documented labels for metric kube_horizontalpodautoscaler_status_desired_replicas are: hpa, namespace
-    main_test.go:112: Metric kube_validatingwebhookconfiguration_metadata_resource_version not found in documentation.
-    main_test.go:112: Metric kube_mutatingwebhookconfiguration_metadata_resource_version not found in documentation.
-    main_test.go:112: Metric kube_node_spec_unschedulable not found in documentation.
-    main_test.go:131: Label horizontalpodautoscaler not found in documentation. Documented labels for metric kube_horizontalpodautoscaler_labels are: hpa, namespace
-FAIL
-FAIL	k8s.io/kube-state-metrics/tests/e2e	0.121s
-FAIL

It shows the following errors, that I will fix in a new commit:

• Metrics kube_pod_start_time and kube_node_spec_unschedulable do not have a stability stage (STABLE, EXPERIMENTAL, etc.)
• For metrics kube_node_status_capacity and kube_node_status_allocatable, in the definition of label unit, characters ` and = have been exchanged.
• Metric kube_namespace_status_phase has a label phase, whereas it is documented as status.
• Metrics kube_deployment_labels and kube_replicaset_labels miss dynamic label label_*
• Metrics kube_horizontalpodautoscaler_* have a label horizontalpodautoscaler, whereas the old label hpa is still documented.
• Documentation files for mutatingwebhookconfiguration and validatingwebhookconfiguration do not end by -metrics.md like other metric documentation files.

/hold

[olivierlemasle]

/assign @tariq1890

[olivierlemasle]

Added commit with documentation fixes; it resolves e2e test failures.
/unhold

[tariq1890]

This is so awesome. Thanks @olivierlemasle :)!

[tariq1890]

Are there any go libraries from stdlib to get the directory the project root?

[olivierlemasle]

Hmm... I suppose we could use https://stackoverflow.com/a/38644571/625158, that uses runtime.Caller, but is it better?

[tariq1890]

I think we can go with that :). From the looks of it, I don't see any problems.

[olivierlemasle]

I've just tested adding:

_, b, _, _ = runtime.Caller(0)

Using make e2e (from the root path of the project), b is actually the absolute path of ./tests/e2e/main_test.go. So I cannot get the documentation path with this method, without using ../../docs/.

[olivierlemasle]

@tariq1890 PTAL thanks!

[tariq1890]

/lgtm

Thanks again @olivierlemasle :)

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: olivierlemasle, tariq1890

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [tariq1890]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment