Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cilium: Hubble fails when cilium_cluster_name is customized. #10247

Closed
kolovo opened this issue Jun 22, 2023 · 2 comments · Fixed by #10614
Closed

Cilium: Hubble fails when cilium_cluster_name is customized. #10247

kolovo opened this issue Jun 22, 2023 · 2 comments · Fixed by #10614
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@kolovo
Copy link

kolovo commented Jun 22, 2023

Environment:

  • Cloud provider or hardware configuration:
    Not Relevant

  • OS (printf "$(uname -srm)\n$(cat /etc/os-release)\n"):
    Not Relevant

  • Version of Ansible (ansible --version):
    Not Relevant

  • Version of Python (python --version):
    Not Relevant

Kubespray version (commit) (git rev-parse --short HEAD):
a1521dc

Network plugin used:
cilium

Full inventory with variables (ansible -i inventory/sample/inventory.ini all -m debug -a "var=hostvars[inventory_hostname]"):
Not Relevant

Command used to invoke ansible:
Not Relevant

Output of ansible run:
Not Relevant

Anything else do we need to know:
When not using the default cluster-name in cilium (cluster.local) hubble fails to start with the following error:
level=warning msg="Failed to create peer client for peers synchronization; will try again after the timeout has expired" error="context deadline exceeded" subsys=hubble-relay target="hubble-peer.kube-system.svc.cluster.local:443"

cilium cluster-name parameter is not set in hubble template(roles/network_plugin/cilium/templates/hubble/config.yml.j2) and therefore fails to start as it is unable to peer the right cluster.
To resolve the issue cluster-name parameter should be added in the template file like below:
cat roles/network_plugin/cilium/templates/hubble/config.yml.j2
apiVersion: v1
kind: ConfigMap
metadata:
name: hubble-relay-config
namespace: kube-system
data:
config.yaml: |
cluster-name: "{{ cilium_cluster_name }}" >> missing parameter
listen-address: :4245
metrics-listen-address: ":9966"
dial-timeout:
...
...

@kolovo kolovo added the kind/bug Categorizes issue or PR as related to a bug. label Jun 22, 2023
@MrFreezeex
Copy link
Member

Hi @kolovo we would be very happy if you could send a PR for this 🙏

@eugene-eeo
Copy link
Contributor

hey @MrFreezeex we're running into this issue on our deployment, so I've made a PR that should fix it: #10614.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants