Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ingress controller installation via helm chart 4.6.0 on AKS 1.26 - not working ingress #9891

Closed
plycloud opened this issue Apr 24, 2023 · 7 comments
Closed

Ingress controller installation via helm chart 4.6.0 on AKS 1.26 - not working ingress #9891

plycloud opened this issue Apr 24, 2023 · 7 comments
Assignees
@longwuyuan
Labels
needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-priority needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.

Comments

@plycloud
Copy link

What happened:

I have installed nginx-ingress controller via helm chart version "4.6.0" with the default values.
The helm chart is deploying ingress controller v1.7.0

What you expected to happen:

Expected behavior is to get ingress controller which is working the same way as deploying it by deployment manifest:
https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.7.0/deploy/static/provider/cloud/deploy.yaml
On the same cluster when ingress controller is deployed via the above manifests all ingress are working.

NGINX Ingress controller version (exec into the pod and run nginx-ingress-controller --version.):

NGINX Ingress controller
Release: v1.7.0
Build: 72ff21e
Repository: https://github.com/kubernetes/ingress-nginx
nginx version: nginx/1.21.6

Kubernetes version (use kubectl version):

WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.3", GitCommit:"9e644106593f3f4aa98f8a84b23db5fa378900bd", GitTreeState:"clean", BuildDate:"2023-03-15T13:40:17Z", GoVersion:"go1.19.7", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.3", GitCommit:"9e644106593f3f4aa98f8a84b23db5fa378900bd", GitTreeState:"clean", BuildDate:"2023-03-21T00:12:01Z", GoVersion:"go1.19.7", Compiler:"gc", Platform:"linux/amd64"}

Environment:
Azure AKS v1.26.3

  • Current State of the controller:

    • kubectl describe ingressclasses
      Name: nginx
      Labels: app.kubernetes.io/component=controller
      app.kubernetes.io/instance=nginx-ingress
      app.kubernetes.io/managed-by=Helm
      app.kubernetes.io/name=ingress-nginx
      app.kubernetes.io/part-of=ingress-nginx
      app.kubernetes.io/version=1.7.0
      helm.sh/chart=ingress-nginx-4.6.0
      Annotations:
      Controller: k8s.io/ingress-nginx
      Events:

    • kubectl -n <ingresscontrollernamespace> describe po <ingresscontrollerpodname>
      Name: nginx-ingress-ingress-nginx-controller-55df7fdf47-bmrxc
      Namespace: ingress-nginx
      Priority: 0
      Service Account: nginx-ingress-ingress-nginx
      Node: aks-gtx-47322197-vmss000000/10.2.0.33
      Start Time: Mon, 24 Apr 2023 15:24:46 +0300
      Labels: app.kubernetes.io/component=controller
      app.kubernetes.io/instance=nginx-ingress
      app.kubernetes.io/managed-by=Helm
      app.kubernetes.io/name=ingress-nginx
      app.kubernetes.io/part-of=ingress-nginx
      app.kubernetes.io/version=1.7.0
      helm.sh/chart=ingress-nginx-4.6.0
      pod-template-hash=55df7fdf47
      Annotations:
      Status: Running
      IP: 10.2.0.34
      IPs:
      IP: 10.2.0.34
      Controlled By: ReplicaSet/nginx-ingress-ingress-nginx-controller-55df7fdf47
      Containers:
      controller:
      Container ID: containerd://f67c54d218abf5c33bd4d007ce31e426b852ca92942317b89fcc848ed73567fd
      Image: registry.k8s.io/ingress-nginx/controller:v1.7.0@sha256:7612338342a1e7b8090bef78f2a04fffcadd548ccaabe8a47bf7758ff549a5f7
      Image ID: registry.k8s.io/ingress-nginx/controller@sha256:7612338342a1e7b8090bef78f2a04fffcadd548ccaabe8a47bf7758ff549a5f7
      Ports: 80/TCP, 443/TCP, 8443/TCP
      Host Ports: 0/TCP, 0/TCP, 0/TCP
      Args:
      /nginx-ingress-controller
      --publish-service=$(POD_NAMESPACE)/nginx-ingress-ingress-nginx-controller
      --election-id=nginx-ingress-ingress-nginx-leader
      --controller-class=k8s.io/ingress-nginx
      --ingress-class=nginx
      --configmap=$(POD_NAMESPACE)/nginx-ingress-ingress-nginx-controller
      --validating-webhook=:8443
      --validating-webhook-certificate=/usr/local/certificates/cert
      --validating-webhook-key=/usr/local/certificates/key
      State: Running
      Started: Mon, 24 Apr 2023 15:24:46 +0300
      Ready: True
      Restart Count: 0
      Requests:
      cpu: 100m
      memory: 90Mi
      Liveness: http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=5
      Readiness: http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
      Environment:
      POD_NAME: nginx-ingress-ingress-nginx-controller-55df7fdf47-bmrxc (v1:metadata.name)
      POD_NAMESPACE: ingress-nginx (v1:metadata.namespace)
      LD_PRELOAD: /usr/local/lib/libmimalloc.so
      Mounts:
      /usr/local/certificates/ from webhook-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-f5dx5 (ro)
      Conditions:
      Type Status
      Initialized True
      Ready True
      ContainersReady True
      PodScheduled True
      Volumes:
      webhook-cert:
      Type: Secret (a volume populated by a Secret)
      SecretName: nginx-ingress-ingress-nginx-admission
      Optional: false
      kube-api-access-f5dx5:
      Type: Projected (a volume that contains injected data from multiple sources)
      TokenExpirationSeconds: 3607
      ConfigMapName: kube-root-ca.crt
      ConfigMapOptional:
      DownwardAPI: true
      QoS Class: Burstable
      Node-Selectors: kubernetes.io/os=linux
      Tolerations: node.kubernetes.io/memory-pressure:NoSchedule op=Exists
      node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
      node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
      Events:
      Type Reason Age From Message

      Normal Scheduled 18m default-scheduler Successfully assigned ingress-nginx/nginx-ingress-ingress-nginx-controller-55df7fdf47-bmrxc to aks-gtx-47322197-vmss000000
      Normal Pulled 18m kubelet Container image "registry.k8s.io/ingress-nginx/controller:v1.7.0@sha256:7612338342a1e7b8090bef78f2a04fffcadd548ccaabe8a47bf7758ff549a5f7" already present on machine
      Normal Created 18m kubelet Created container controller
      Normal Started 18m kubelet Started container controller
      Normal RELOAD 18m nginx-ingress-controller NGINX reload triggered due to a change in configuration

    • kubectl -n <ingresscontrollernamespace> describe svc <ingresscontrollerservicename>
      Name: nginx-ingress-ingress-nginx-controller
      Namespace: ingress-nginx
      Labels: app.kubernetes.io/component=controller
      app.kubernetes.io/instance=nginx-ingress
      app.kubernetes.io/managed-by=Helm
      app.kubernetes.io/name=ingress-nginx
      app.kubernetes.io/part-of=ingress-nginx
      app.kubernetes.io/version=1.7.0
      helm.sh/chart=ingress-nginx-4.6.0
      Annotations:
      Selector: app.kubernetes.io/component=controller,app.kubernetes.io/instance=nginx-ingress,app.kubernetes.io/name=ingress-nginx
      Type: LoadBalancer
      IP Family Policy: SingleStack
      IP Families: IPv4
      IP: 10.2.73.172
      IPs: 10.2.73.172
      LoadBalancer Ingress: 20.50.98.208
      Port: http 80/TCP
      TargetPort: http/TCP
      NodePort: http 30589/TCP
      Endpoints: 10.2.0.34:80
      Port: https 443/TCP
      TargetPort: https/TCP
      NodePort: https 30896/TCP
      Endpoints: 10.2.0.34:443
      Session Affinity: None
      External Traffic Policy: Cluster
      Events:
      Type Reason Age From Message

      Normal EnsuringLoadBalancer 20m service-controller Ensuring load balancer
      Normal EnsuredLoadBalancer 20m service-controller Ensured load balancer

  • Current state of ingress object, if applicable:
    All ingresses are in state: nginx-ingress-controller Scheduled for sync
    All ingresses are shown with the LB address.
    None of the application URLs is accessible.

  • Others:
    Using mergable ingress annotations:
    nginx.org/mergeable-ingress-type: master
    and
    nginx.org/mergeable-ingress-type: minion
@plycloud plycloud added the kind/bug Categorizes issue or PR as related to a bug. label Apr 24, 2023
@k8s-ci-robot k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Apr 24, 2023
@k8s-ci-robot
Copy link
Contributor

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@longwuyuan
Copy link
Contributor

/remove-kind bug

@plycloud its not easy to test all the supported clouds and environments in CI and you can get a diff in the manifest using helm template.

Also, you can discuss on kubernetes.slack.com as there are lots of users and experts there. Register at slack.k8s.io if required.

@k8s-ci-robot k8s-ci-robot added needs-kind Indicates a PR lacks a `kind/foo` label and requires one. and removed kind/bug Categorizes issue or PR as related to a bug. labels Apr 24, 2023
@longwuyuan
Copy link
Contributor

/assign

@plycloud
Copy link
Author

I have found the solution for AKS:

controller:
service:
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
externalTrafficPolicy: Local

Those options must be set in order ingresses to work.
Same options are set by default in deployment manifest https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.7.0/deploy/static/provider/cloud/deploy.yaml

@longwuyuan
Copy link
Contributor

/close

@k8s-ci-robot
Copy link
Contributor

@longwuyuan: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@fgielow
Copy link

fgielow commented Jul 18, 2023

I have found the solution for AKS:

controller:
service:
ipFamilies:

  • IPv4
    ipFamilyPolicy: SingleStack
    externalTrafficPolicy: Local

Those options must be set in order ingresses to work. Same options are set by default in deployment manifest https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.7.0/deploy/static/provider/cloud/deploy.yaml

Just reporting I also required to add these manually as otherwise I suffered from the same issues described.

@longwuyuan longwuyuan mentioned this issue Jan 19, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@longwuyuan longwuyuan

Labels
needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-priority needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one.
Projects
[SIG Network] Ingress NGINX
Archived in project
Milestone
No milestone
Development

No branches or pull requests
4 participants
@fgielow @longwuyuan @k8s-ci-robot @plycloud