Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

port 80 is free, but nginx-ingress-controller say already in use #6988

Closed
boxjan opened this issue Mar 24, 2021 · 6 comments
Closed

port 80 is free, but nginx-ingress-controller say already in use #6988

boxjan opened this issue Mar 24, 2021 · 6 comments
Labels
lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. triage/needs-information Indicates an issue needs more information in order to work on it.

Comments

@boxjan
Copy link
Contributor

boxjan commented Mar 24, 2021

NGINX Ingress controller version: v0.44.0

Kubernetes version (use kubectl version):

Environment:

  • Cloud provider or hardware configuration: 2C4G
  • OS (e.g. from /etc/os-release): Ubuntu 20.04
  • Kernel (e.g. uname -a): 5.4.0-70-generic Fix #76: hardcode X-Forwarded-Port due to SSL Passthrough #78-Ubuntu SMP Fri Mar 19 13:29:52 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools: cilium
  • Others: deploy as DaemonSet in cluster

What happened:
After I restart one node, the ingress can not up because 80 have in used; err log is here

-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v0.44.0
  Build:         f802554ccfadf828f7eb6d3f9a9333686706d613
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.19.6

-------------------------------------------------------------------------------

I0324 15:25:08.612896       7 flags.go:208] "Watching for Ingress" class="nginx"
W0324 15:25:08.613037       7 flags.go:213] Ingresses with an empty class will also be processed by this Ingress controller
F0324 15:25:08.614516       7 main.go:68] port 80 is already in use. Please check the flag --http-port
goroutine 1 [running]:
k8s.io/klog/v2.stacks(0xc0000b0001, 0xc000434000, 0x67, 0xa4)
        k8s.io/klog/[email protected]/klog.go:1026 +0xb9
k8s.io/klog/v2.(*loggingT).output(0x26915e0, 0xc000000003, 0x0, 0x0, 0xc0003f1c00, 0x25e6c33, 0x7, 0x44, 0x40e200)
        k8s.io/klog/[email protected]/klog.go:975 +0x19b
k8s.io/klog/v2.(*loggingT).printDepth(0x26915e0, 0xc000000003, 0x0, 0x0, 0x0, 0x0, 0x1, 0xc00004d380, 0x1, 0x1)
        k8s.io/klog/[email protected]/klog.go:732 +0x16f
k8s.io/klog/v2.(*loggingT).print(...)
        k8s.io/klog/[email protected]/klog.go:714
k8s.io/klog/v2.Fatal(...)
        k8s.io/klog/[email protected]/klog.go:1482
main.main()
        k8s.io/ingress-nginx/cmd/nginx/main.go:68 +0x1fe

goroutine 18 [chan receive]:
k8s.io/klog/v2.(*loggingT).flushDaemon(0x26915e0)
        k8s.io/klog/[email protected]/klog.go:1169 +0x8b
created by k8s.io/klog/v2.init.0
        k8s.io/klog/[email protected]/klog.go:417 +0xdf

What you expected to happen:

After reboot a node and apiserver up, the ingress also ready to work.

How to reproduce it:

My cluster has 4 nodes, 3 of which are masters, and I used https://docs.cilium.io/en/stable/gettingstarted/kubeproxy-free/ to set up k8s with cilium and setup DSR; later used https://github.com/kubernetes/ingress-nginx/tree/master/charts/ingress-nginx deployed ingress-nginx
When the services are all ready, restart one of the nodes, but after the node restarts, all the original components can be started normally, except for ingress-nginx;

I use NodePort to allow internet visit the ingress.

I used a debug pod just like kubectl -n nginx debug nginx-ingress-controller-2snsf -it --image=k8s.gcr.io/ingress-nginx/controller:v0.44.0 --share-processes --copy-to=nginx-ingress-controller-2snsf-debug -- bash try to found what happened.

When I use the netstat -lnt to see any application listen on 80/443, but nothing here. But when i use curl 0.0.0.0:80 -vvvv to get anything, and get other nginx pod response.

bash-5.1$ netstat -lnt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
bash-5.1$ curl 0.0.0.0:80 -vvvv
*   Trying 0.0.0.0:80...
* Connected to 0.0.0.0 (10.192.0.237) port 80 (#0)
> GET / HTTP/1.1
> Host: 0.0.0.0
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< Date: Wed, 24 Mar 2021 15:49:51 GMT
< Content-Type: text/html
< Content-Length: 146
< Connection: keep-alive
<
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host 0.0.0.0 left intact
bash-5.1$

It seem that the port have been connet by a tunnel.

I read the code and notice https://github.com/kubernetes/ingress-nginx/blob/master/internal/net/net.go#L31
the function try to connect the port to ensure nginx can listen on it. I think maybe try to bind the port, or we got the problem here.

Anything else we need to know:

English is not my mother language, sorry for that.
/kind bug
@boxjan boxjan added the kind/bug Categorizes issue or PR as related to a bug. label Mar 24, 2021
@boxjan boxjan mentioned this issue Mar 25, 2021
Merged
8 tasks
@longwuyuan
Copy link
Contributor

/remove-kind bug
/triage needs-information

Please show ;

  • kubectl get all -A -o wide
  • kubectl -n logs

@k8s-ci-robot k8s-ci-robot added triage/needs-information Indicates an issue needs more information in order to work on it. and removed kind/bug Categorizes issue or PR as related to a bug. labels Apr 27, 2021
@k8s-triage-robot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 27, 2021
@tao12345666333
Copy link
Member

#6990 has been merged. let's close this one.

/close

@k8s-ci-robot
Copy link
Contributor

@tao12345666333: Closing this issue.

In response to this:

#6990 has been merged. let's close this one.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@chessman chessman mentioned this issue Aug 9, 2021
Merged
8 tasks
@aureq aureq mentioned this issue Oct 30, 2021
Closed
@qianlei360
Copy link

recurrent in v1.1.0
image

@jiaminxu
Copy link

recurrent in v1.1.0 image

i have this bug in 1.1.1 ，but i fix it by apply deploy with 1.2.1 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. triage/needs-information Indicates an issue needs more information in order to work on it.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@tao12345666333 @longwuyuan @k8s-ci-robot @jiaminxu @boxjan @k8s-triage-robot @qianlei360