Add support for hot-reloading on CRS config changes #1892
Assignees
Labels
kind/feature
Categorizes issue or PR as related to a new feature.
Comments
rexagod
added
the
kind/feature
|
/assign |
mrueg
added a commit
to mrueg/kube-state-metrics
that referenced
this issue
Dec 8, 2022
This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. Code for this file change detection was reused from https://github.com/spf13/viper licensed under MIT license. Fixes: kubernetes#1892
mrueg
added a commit
to mrueg/kube-state-metrics
that referenced
this issue
Dec 8, 2022
This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. It also resolves a bug in which customresourcestatemetrics were not added when resources where non-default resources. Code for this file change detection was reused from https://github.com/spf13/viper licensed under MIT license. Fixes: kubernetes#1892
mrueg
added a commit
to mrueg/kube-state-metrics
that referenced
this issue
Dec 8, 2022
This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. It also resolves a bug in which customresourcestatemetrics were not added when resources where non-default resources. Code for this file change detection was reused from https://github.com/spf13/viper licensed under MIT license. Fixes: kubernetes#1892
mrueg
added a commit
to mrueg/kube-state-metrics
that referenced
this issue
Dec 8, 2022
This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. It also resolves a bug in which customresourcestatemetrics were not added when resources where non-default resources. Code for this file change detection was reused from https://github.com/spf13/viper licensed under MIT license. Fixes: kubernetes#1892
mrueg
added a commit
to mrueg/kube-state-metrics
that referenced
this issue
Dec 8, 2022
This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. It also resolves a bug in which customresourcestatemetrics were not added when set resources were non-default resources. Code for this file change detection was reused from https://github.com/spf13/viper licensed under MIT license. Fixes: kubernetes#1892
mrueg
added a commit
to mrueg/kube-state-metrics
that referenced
this issue
Dec 8, 2022
This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. It also resolves a bug in which customresourcestatemetrics were not added when set resources were non-default resources. Code for this file change detection was reused from https://github.com/spf13/viper licensed under MIT license. Fixes: kubernetes#1892
mrueg
added a commit
to mrueg/kube-state-metrics
that referenced
this issue
Dec 8, 2022
This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. It also resolves a bug in which customresourcestatemetrics were not added when set resources were non-default resources. Code for this file change detection was reused from https://github.com/spf13/viper licensed under MIT license. Fixes: kubernetes#1892
mrueg
added a commit
to mrueg/kube-state-metrics
that referenced
this issue
Dec 8, 2022
This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. It also resolves a bug in which customresourcestatemetrics were not added when set resources were non-default resources. Code for this file change detection was reused from https://github.com/spf13/viper licensed under MIT license. Fixes: kubernetes#1892
mrueg
added a commit
to mrueg/kube-state-metrics
that referenced
this issue
Dec 17, 2022
This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. It also resolves a bug in which customresourcestatemetrics were not added when set resources were non-default resources. Code for this file change detection was reused from https://github.com/spf13/viper licensed under MIT license. Fixes: kubernetes#1892
mrueg
added a commit
to mrueg/kube-state-metrics
that referenced
this issue
Jan 9, 2023
This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. It also resolves a bug in which customresourcestatemetrics were not added when set resources were non-default resources. Code for this file change detection was reused from https://github.com/spf13/viper licensed under MIT license. Fixes: kubernetes#1892
mrueg
added a commit
to mrueg/kube-state-metrics
that referenced
this issue
Jan 9, 2023
This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. It also resolves a bug in which customresourcestatemetrics were not added when set resources were non-default resources. Fixes: kubernetes#1892
subbarao-meduri
pushed a commit
to stolostron/kube-state-metrics
that referenced
this issue
Jun 1, 2023
* add rolebinding metrics * fix typo * fix filename and sort order * Apply suggestions from code review Co-authored-by: Manuel Rüger <[email protected]> * fix metrics name * fix typos * feat(logging): migrate to structured logging Signed-off-by: dmpe <John Malc> <[email protected]> * Apply suggestions from code review Co-authored-by: JUN YANG <[email protected]> * Apply suggestions from code review Co-authored-by: JUN YANG <[email protected]> * Apply suggestions from code review Co-authored-by: JUN YANG <[email protected]> * Apply suggestions from code review Co-authored-by: JUN YANG <[email protected]> * Replace pkg/errors with stdlib errors github.com/pkg/errors is archived and not maintained anymore. Replace it with stdlib functions. Signed-off-by: Manuel Rüger <[email protected]> * pkg/customresourcestate implement info and stateSet metric type and refactor configuration file * Adds detection of booleans in string format to getNum. * Refactors configuration file to allow definition of different metric types having different configuration variables. * Refactor order of types and funcs in pkg/customersourcestate. * Allows info and stateSet metrics to iterate over arrays. * Adds `nilIsZero` config variable to gauge to indicate non-existing values to tread as 0 value instead of returning an error. * Skip adding a label instead of setting value to `<nil>`. * Replace namespace and subsystem by metricsNamePrefix * Adjust docs for customresourcestate metrics to align with new configuration file * Do not expose info metric for nil objects * Update dependencies go v1.18.3 -> v1.18.5 prometheus v2.35.0 -> v2.37.0 golangci-lint v1.46.2 -> v1.48.0 Several go dependencies, among them: k8s v1.24.2 -> v1.24.4 cloudbuild image v20211118-2f2d816b90 -> v20220609-2e4c91eb7e Signed-off-by: Manuel Rüger <[email protected]> * graduate new endpoint metrics to STABLE * graduate kube_endpoint_ports and kube_endpoint_address to STABLE * graduate kube_endpoint_address_not_ready and kube_endpoint_address_available to DEPRECATED as the information is precomputed during metrics-scraping Signed-off-by: Mario Constanti <[email protected]> * feat: Add local storage labels to kube_persistentvolume_info Signed-off-by: m.nabokikh <[email protected]> * Make code inline with the other parts Signed-off-by: m.nabokikh <[email protected]> * Remove RBAC resources from default exposed metric set These resources might create a lot of metrics, so we won't enable them by default for now. Signed-off-by: Manuel Rüger <[email protected]> * Prevent multiple custom resource configurations for the same resource * Add host path metrics, fix table formatting Signed-off-by: m.nabokikh <[email protected]> * *: Cut v2.6.0 * add exit code * fix unitests * create new metric * unit tests * remove reason from exitcode * fix test * documentation * Promote two metrics to stable * Update ci.yml Signed-off-by: sashashura <[email protected]> * add ContainerResourceSourceType for hpa metrics and reduce cyclomatic complexity * fix/docs: Typo correction in SA metrics Typo correction in SA metrics documentation in the summary * Do not expose ingress path metric when service is nil * Fix return blank string when Service is nil * Fix none to blank string * Handle singular labels in allowlist Handle singular labels in allowlist failing when such a label is supplied, in order to keep the behaviour in sync with --resources. Signed-off-by: Pranshu Srivastava <[email protected]> * Allow Lease metrics to be exported across all namespaces * update lease store to export namespace * go.mod: Upgrade to k8s 1.25 Signed-off-by: Manuel Rüger <[email protected]> * Build with go 1.19 Signed-off-by: Manuel Rüger <[email protected]> * README.md: Replace Kubernetes compat matrix Signed-off-by: Manuel Rüger <[email protected]> * Represent GVK information as labels Represent GVK information as labels in the metrics, instead of appending them to the metric name itself. This would allow users to aggregate varying GVKs of a CR under the same metric, making operations much more easier. * export Lease.Spec.HolderIdentity * account for nil holderIdentity case * Import k8s metrics stability framework * Upgrade dependency component-base to v0.25.2 and add TODO * Add all stable metrics * customresourcestate fix type indentation in example * fixup! Represent GVK information as labels * fixup! fixup! Represent GVK information as labels * Harden and add gosec linter Remediate: G104: Errors unhandled. G109: Potential Integer overflow made by strconv.Atoi result conversion to int16/32 G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server G304: Potential file inclusion via variable G601: Implicit memory aliasing in for loop. Signed-off-by: Manuel Rüger <[email protected]> * .github: Update actions Context: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/ Signed-off-by: Manuel Rüger <[email protected]> * add myself to OWNERs * Fix typos * One more typo fix * Clarify valueFrom comment * e2e: Test against k8s 1.25.0 Signed-off-by: Manuel Rüger <[email protected]> * Makefile: Build with go 1.19.3 Signed-off-by: Manuel Rüger <[email protected]> * go.mod: Bump dependencies Signed-off-by: Manuel Rüger <[email protected]> * Update OWNERS * Recommend kube-scheduler alternatives Recommend metrics exposed by the kube-scheduler, in case of the ones below: * `kube_pod_container_resource_limits` * `kube_pod_container_resource_requests` Signed-off-by: Pranshu Srivastava <[email protected]> * Allow `labelFromKey` field Allow `labelFromKey` field for the following types: * Gauge: Done. * Info: Done. * StateSet: N/A (redundant use case, see doc changes for more info). Signed-off-by: Pranshu Srivastava <[email protected]> * Address vulns reported by the security checks As of this moment, there are 2 vulns in the codebase. * GO-2022-1095: os/[email protected] * GO-2022-1095: [email protected] This commit aims to fix that. Signed-off-by: Pranshu Srivastava <[email protected]> * Sharding per node * Validate options * Clean * Move merging fieldselectors into app/server.go and replace namespaceFitler with fieldSelectorFilter * Refactoring * Provide scaling example * Add `govulncheck` cron Add `govulncheck` cron configuration. Signed-off-by: Pranshu Srivastava <[email protected]> * Update README.md Co-authored-by: Manuel Rüger <[email protected]> * Refactoring * introduce custom-resources-only flag ... ... to only monitor all known custom-resource configurations instead of listing each of them explicitly Signed-off-by: Mario Constanti <[email protected]> * Support filtering label allowlist by "*" Support filtering label allowlist by "*", which will expand to the enabled resources, while infering their values based on its value(s). Signed-off-by: Pranshu Srivastava <[email protected]> * Add retention policy metrics for KEP-1847 * fixup! Allow `labelFromKey` field * fixup! fixup! Allow `labelFromKey` field * fixup! fixup! fixup! Allow `labelFromKey` field * Deprecate VPA Deprecate VPA metrics in v2.9.0. Signed-off-by: Pranshu Srivastava <[email protected]> * fixup! fixup! fixup! fixup! Allow `labelFromKey` field * go.mod: Bump exporter-toolkit to 0.8.1 Signed-off-by: Manuel Rüger <[email protected]> * Incorporate Cobra s/pflags/cobra/g: * Use spf13/cobra to handle all flags and sub-commands. * Remove all spf13/pflag usage, and fallback to the in-build flag package if, and when needed. * Add completion support. Signed-off-by: Pranshu Srivastava <[email protected]> * update kube_node_status_{capacity/allocatable} doc to clarify difference * Add node deletionTimestamp metric Adds deletionTimestamp metric, for nodes. Signed-off-by: Pranshu Srivastava <[email protected]> * Add --config flag --config flag defines the path to the kube-state-metrics options config file. * Implement hot-reloading based on config changes Reload KSM on-the-fly when a change is detected in the configuration. Meta changes (will squash) * Add end-to-end testing to verify hot-reloading Add end-to-end testing to verify hot-reloading for event-based changes stemming from the config file. Also, * sent in a doc fix that was missed earlier: https://github.com/kubernetes/kube-state-metrics/pull/1890/files#diff-380eca5a922c0ddbf67f04daefc6823e7ef0e197434d3a826d39c7063cdfa6d6R15, * updated fsnotify and viper dependencies (v1.6.0 and v1.14.0 respectively). Signed-off-by: Pranshu Srivastava <[email protected]> * Add rexagod as reviewer Signed-off-by: Damien Grisonnet <[email protected]> * add ingress class metrics * add ingress classs as nondefault resource * add stability experimental * autoscaling/v2beta2 HorizontalPodAutoscaler is deprecated in v1.23+ Issue: kubernetes#1711 Problem: kube-state-metrics supports up to (latest k8s release - 3) k8s version. Since v1.25 has been release we can update it to start using autoscaler/v2. Solution: update packages to start using autoscaler/v2 Signed-off-by: JoaoBraveCoding <[email protected]> * Update internal/store/builder.go Co-authored-by: Damien Grisonnet <[email protected]> * fix linter error * go.mod: Update dependencies * *: Cut v2.7.0 Signed-off-by: Manuel Rüger <[email protected]> * Add Metrics for EndpointSlices Implements https://pkg.go.dev/k8s.io/api/discovery/v1#EndpointSlice This resourcetype is disabled by default as they are very verbose and have a high cardinality. Metrics from endpointslices can be used to identify if specific pods are part of an endpoint and thus discoverable through a service. Signed-off-by: Manuel Rüger <[email protected]> * Add CatherineF-dev as reviewer Signed-off-by: Damien Grisonnet <[email protected]> * Update github.com/prometheus/exporter-toolkit The latest version (v0.8.2) fixes CVE-2022-46146. Signed-off-by: Simon Pasquier <[email protected]> * Add metrics for config file changes This uses code pieces from prometheus/alertmanager in https://github.com/prometheus/alertmanager/blob/main/config/coordinator.go#LL56C26-L56C26 licensed under Apache-2.0. kube_state_metrics_config_hash{type="config", filename="config.yml"} 4.0061079457904e+13 kube_state_metrics_config_last_reload_success_timestamp_seconds{type="config", filename="config.yml"} 1.6697483049487052e+09 kube_state_metrics_config_last_reload_successful{type="config", filename="config.yml"} 1 Signed-off-by: Manuel Rüger <[email protected]> * Replace "<none>" with empty string for "owner_kind", "owner_name" and "owner_is_controller" dimensions. Returning empty string instead of "<none>" value for "owner_kind", "owner_name" and "owner_is_controller" dimensions when no metadata.ownerReferences exists in Kubernetes resoures. * Replace special string "<none>" with empty string for "storageclass" dimension. Returning empty string instead of special string "<none>" for "storageclass" dimensions of "kube_persistentvolumeclaim_info" metric. * Update internal/store/persistentvolumeclaim.go Co-authored-by: Manuel Rüger <[email protected]> * Remove duplicated MetricsWriter implementation Simplify the implementation of the MetricsWriter to avoid code duplication between single and multi stores scenarios. Signed-off-by: Damien Grisonnet <[email protected]> * pkg/metrics_store: add error handling to WriteAll Signed-off-by: Damien Grisonnet <[email protected]> * Add metrics for CustomResourceConfig file * docs: add status badge for the periodic cronjob Additionally, rename it to a more fitting name. * Rename references from master to main branch * Add kube_pod_status_qos_class gauge to pod metrics Signed-off-by: frezes <[email protected]> * Makefile: Bump to go 1.19.4 * Makefile: Bump to prometheus 2.40.6 * go.mod: Bump to kubernetes 1.26.0 * .github: Bump golangci-lint to 1.50.1 * Make CRS metrics type dynamic All CRS metrics are hardcoded to "gauge" type, this patch addresses that. * Add develop doc on adding new metrics * Update docs/developer/guide.md Co-authored-by: Pranshu Srivastava <[email protected]> * Change kube_pod_status_qos_class to experimental metric Signed-off-by: frezes <[email protected]> * docs: Fix typo in kube_node_deletion_timestamp description Signed-off-by: Grzegorz Głąb <[email protected]> * docs:fix link * prefix GVK labels in CustomResourceMonitoring This will prefix the auto-generated GVK labels for CustomResources with customresource_ to make it a bit more clear that these labels got generated. Signed-off-by: Mario Constanti <[email protected]> * Reload CustomResourceState Config File on Change This change adds hot reloading support for the customresourcestate config file. It also resolves a bug in which the customresourcestate config file was included in the ksm config file, in which it did not get detected. It also resolves a bug in which customresourcestatemetrics were not added when set resources were non-default resources. Fixes: kubernetes#1892 * replace kube_crd with kube_customresource in docs Signed-off-by: Mario Constanti <[email protected]> * fix vpa crd metric names * update crd monitoring and mention new flags * Support pod_ready_time and pod_container_ready_time Co-authored-by: Szymon Grzemski <[email protected]> Signed-off-by: Lan Liang <[email protected]> * Addressed feedback * Fixing bad merge in rebase * Update development guide * fix --version flag * Added a test for kube_pod_status_container_ready_time * tools: Use own go.mod This reduces the number of deps that will be fetched for building ksm * clean broken --version flag * remove unwanted change * update doc * Enhance UserAgent with more information Before: "User-Agent: v2.7.0" After: "User-Agent: kube-state-metrics/v2.7.0 (linux/amd64) kubernetes/1cda0bf9" * Handle unit length `valueFrom` values Handle unit length `valueFrom` values and skip strings where we expect them to be type-cast-able to `float64`, instead of erroring, since that is the expected behavior, and what's being done for other types. * support "True" and "False" as string in custom-resource-state for operator status conditions * lowercase string and simplify logic Co-authored-by: Manuel Rüger <[email protected]> * Improve command to install tools * Document how gauges convert types to float * Improve formating * Use structured logging * Fixing emitting of ready time metrics when condition is false * Set OpenMetrics content header See: https://github.com/prometheus/common/blob/main/expfmt/encode.go#L86 See: kubernetes#1973 * Migrate all NewFamilyGenerator to NewFamilyGeneratorWithStability * Clean up NewFamilyGenerator in comments * Update docs/customresourcestate-metrics.md * fixup! Handle unit length `valueFrom` values * Update docs/customresourcestate-metrics.md * go.mod: Update dependencies * *: Cut v2.8.0 * fix: public Builder compatibility with the BuilderInterface * Don't crash on non-existent path values Don't crash on non-existent path values in CRS. Signed-off-by: Pranshu Srivastava <[email protected]> * fixup! Don't crash on non-existent path values * build: Bump to go 1.19.6 * *: Cut v2.8.1 * Only use OpenMetrics and Text in contentType * Update pkg/metricshandler/metrics_handler.go Co-authored-by: Manuel Rüger <[email protected]> * Update golangci-lint version * Update golang and go deps Signed-off-by: Manuel Rüger <[email protected]> * *: Cut v2.8.2 * checked out upstream upgrade tag and replayed custom changes on top Signed-off-by: Nathaniel Graham <[[email protected], [email protected]]> * removed travis CI yaml file Signed-off-by: Nathaniel Graham <[[email protected], [email protected]]> * updated Dockerfile.prow to try go1.19 Signed-off-by: Nathaniel Graham <[[email protected], [email protected]]> * updated go.mod to specify go1.20 Signed-off-by: Nathaniel Graham <[[email protected], [email protected]]> --------- Signed-off-by: dmpe <John Malc> <[email protected]> Signed-off-by: Manuel Rüger <[email protected]> Signed-off-by: Mario Constanti <[email protected]> Signed-off-by: m.nabokikh <[email protected]> Signed-off-by: sashashura <[email protected]> Signed-off-by: Pranshu Srivastava <[email protected]> Signed-off-by: Damien Grisonnet <[email protected]> Signed-off-by: JoaoBraveCoding <[email protected]> Signed-off-by: Simon Pasquier <[email protected]> Signed-off-by: frezes <[email protected]> Signed-off-by: Grzegorz Głąb <[email protected]> Signed-off-by: Lan Liang <[email protected]> Signed-off-by: Nathaniel Graham <[[email protected], [email protected]]> Co-authored-by: Kubernetes Prow Robot <[email protected]> Co-authored-by: Kaito Ii <[email protected]> Co-authored-by: Manuel Rüger <[email protected]> Co-authored-by: dmpe <[email protected]> Co-authored-by: JUN YANG <[email protected]> Co-authored-by: Christian Schlotter <[email protected]> Co-authored-by: Mario Constanti <[email protected]> Co-authored-by: m.nabokikh <[email protected]> Co-authored-by: Shaun Sabo <[email protected]> Co-authored-by: Catherine Fang <[email protected]> Co-authored-by: Alex <[email protected]> Co-authored-by: whitebear009 <[email protected]> Co-authored-by: Quentin DUPUY <[email protected]> Co-authored-by: evir35 <[email protected]> Co-authored-by: Pranshu Srivastava <[email protected]> Co-authored-by: Lanting Chiang <[email protected]> Co-authored-by: Han Kang <[email protected]> Co-authored-by: Pavel Timofeev <[email protected]> Co-authored-by: Matthew Cary <[email protected]> Co-authored-by: Akshit Tyagi <[email protected]> Co-authored-by: Damien Grisonnet <[email protected]> Co-authored-by: JoaoBraveCoding <[email protected]> Co-authored-by: Damien Grisonnet <[email protected]> Co-authored-by: Simon Pasquier <[email protected]> Co-authored-by: Paweł Kubica <[email protected]> Co-authored-by: Paweł Kubica <[email protected]> Co-authored-by: frezes <[email protected]> Co-authored-by: Pranshu Srivastava <[email protected]> Co-authored-by: Grzegorz Głąb <[email protected]> Co-authored-by: yosshi825 <[email protected]> Co-authored-by: Lan Liang <[email protected]> Co-authored-by: Szymon Grzemski <[email protected]> Co-authored-by: Ryan Olds <[email protected]> Co-authored-by: Ryan R. Olds <[email protected]> Co-authored-by: Benjamin Jorand <[email protected]> Co-authored-by: Jan Kantert <[email protected]> Co-authored-by: jabdoa2 <[email protected]> Co-authored-by: Cedric Lamoriniere <[email protected]> Co-authored-by: Artur Rodrigues <[email protected]> Co-authored-by: Nathaniel Graham <[[email protected], [email protected]]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What would you like to be added: Hot-reloading when CRS config changes are detected.
Why is this needed: Allow users to make changes to their config and have KSM pick up on that automatically.
Describe the solution you'd like: Extend #1827 for CRS config.
The text was updated successfully, but these errors were encountered: