Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] kube-ovn-controller cannot be ready when ENABLE_METRICS is false #4862

Open
zhaocongqi opened this issue Dec 23, 2024 · 3 comments · May be fixed by #4886
Open

[BUG] kube-ovn-controller cannot be ready when ENABLE_METRICS is false #4862

zhaocongqi opened this issue Dec 23, 2024 · 3 comments · May be fixed by #4886
Labels
bug Something isn't working

Comments

@zhaocongqi
Copy link
Member

Kube-OVN Version

v1.14.0

Kubernetes Version

v1.29.5

Operation-system/Kernel Version

"Ubuntu 22.04.5 LTS" 6.8.0-50-generic

Description

1、当关闭metrics high kube-ovn-controller的pod不能ready
2、kube-ovn-controller通过10660端口进行健康检查
image
3、但是当ENABLE_METRICS 设置成false时,conteoller不会开启10660端口的监听
image
4、但是在cni的逻辑中,即使ENABLE_METRICS 设置成false,也会开启10665的监听
image

Steps To Reproduce

1、helm chart设置ENABLE_METRICS 为false
2、启动kubeovn,发现controller无法启动

Current Behavior

controller健康检查失败

Expected Behavior

controller健康检查在关闭metric的情况下能够通过

@zhaocongqi zhaocongqi added the bug Something isn't working label Dec 23, 2024
Copy link

dosubot bot commented Dec 23, 2024

The issue you're experiencing is related to the kube-ovn-controller not being ready when ENABLE_METRICS is set to false. This is because the health check for the controller relies on port 10660, which is not opened when metrics are disabled. The readiness probe is configured to use this port for health checks, and if the port is not open, the probe will fail, causing the controller to be marked as not ready.

In contrast, the CNI server logic opens port 10665 even when ENABLE_METRICS is set to false, which is why it does not face the same issue.

To resolve this, you would need to modify the controller's logic to ensure that the health check port (10660) is opened regardless of the ENABLE_METRICS setting, similar to how the CNI server handles its port. This would involve changing the conditional logic in the controller's code to allow the health check server to run independently of the metrics server.

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other

@zhaocongqi
Copy link
Member Author

image
创建server的时候会校验ExtraHandlers是否覆盖了默认的路径 “/metrics”

@zhaocongqi
Copy link
Member Author

image
start 会默认监听 “/metrics”

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant