Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use debian-base-$(ARCH):v2.0.0 as base image #370

Merged
merged 1 commit into from
Apr 15, 2020
Merged

Use debian-base-$(ARCH):v2.0.0 as base image #370

merged 1 commit into from
Apr 15, 2020

Conversation

champtar
Copy link
Contributor

@champtar champtar commented Apr 13, 2020 •
edited
Loading

Use --no-install-recommends to not install nftables as it depends on bash
(https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956655)

Only build tested for amd64 for now
champtar/k8s-dns-sidecar-amd64 1.15.11-14-g423a2b4 b28aafaa2996
champtar/k8s-dns-node-cache-amd64 1.15.11-14-g423a2b4 bb1e2b6e0b09
champtar/k8s-dns-kube-dns-amd64 1.15.11-14-g423a2b4 943afc53b468
champtar/k8s-dns-dnsmasq-nanny-amd64 1.15.11-14-g423a2b4 28526da70d44
champtar/k8s-dns-dnsmasq-amd64 1.15.11-14-g423a2b4 a13a493a9883

Edit: update images

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 13, 2020
@k8s-ci-robot
Copy link
Contributor

Hi @champtar. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Apr 13, 2020
@k8s-ci-robot k8s-ci-robot requested review from bowei and prameshj April 13, 2020 23:20
@champtar
Copy link
Contributor Author

/assign @prameshj

@champtar champtar mentioned this pull request Apr 13, 2020
Merged
@prameshj
Copy link
Contributor

Thanks for sending this out! Looks good to me. Would you mind pasting the output of:

./image-checks.sh 1.15.11-14-g8f51218 champtar

@champtar
Copy link
Contributor Author

I need to add the ARCH

diff --git a/image-checks.sh b/image-checks.sh
index b3056c8..9f29f74 100755
--- a/image-checks.sh
+++ b/image-checks.sh
@@ -5,13 +5,14 @@
 # Kill with Ctrl + C once sidecar starts up successfully.
 TAG=$1
 REGISTRY=${2:-gcr.io/google-containers}
+ARCH=${3:-amd64}
 echo "Verifying that iptables exists in node-cache image"
-docker run --rm -it --entrypoint=iptables ${REGISTRY}/k8s-dns-node-cache:${TAG}
+docker run --rm -it --entrypoint=iptables ${REGISTRY}/k8s-dns-node-cache-${ARCH}:${TAG}
 echo "Verifying that node-cache binary exists in node-cache image"
-docker run --rm -it --entrypoint=/node-cache ${REGISTRY}/k8s-dns-node-cache:${TAG}
+docker run --rm -it --entrypoint=/node-cache ${REGISTRY}/k8s-dns-node-cache-${ARCH}:${TAG}
 echo "Verifying dnsmasq-nanny startup"
-docker run --rm -it --entrypoint=/dnsmasq-nanny ${REGISTRY}/k8s-dns-dnsmasq-nanny:${TAG}
+docker run --rm -it --entrypoint=/dnsmasq-nanny ${REGISTRY}/k8s-dns-dnsmasq-nanny-${ARCH}:${TAG}
 echo "Verifying kube-dns startup"
-docker run --rm -it --entrypoint=/kube-dns ${REGISTRY}/k8s-dns-kube-dns:${TAG}
+docker run --rm -it --entrypoint=/kube-dns ${REGISTRY}/k8s-dns-kube-dns-${ARCH}:${TAG}
 echo "Verifying sidecar startup"
-docker run --rm -it --entrypoint=/sidecar ${REGISTRY}/k8s-dns-sidecar:${TAG}
+docker run --rm -it --entrypoint=/sidecar ${REGISTRY}/k8s-dns-sidecar-${ARCH}:${TAG}

Here the output:

./image-checks.sh 1.15.11-14-g8f51218 champtar
Verifying that iptables exists in node-cache image
iptables v1.8.2 (nf_tables): no command specified
Try `iptables -h' or 'iptables --help' for more information.
Verifying that node-cache binary exists in node-cache image
2020/04/14 02:53:42 [FATAL] Error parsing flags - Invalid localip specified - "", Exiting
Verifying dnsmasq-nanny startup
E0414 02:53:44.611416       1 nanny.go:154] Error reading from stderr: read |0: file already closed
F0414 02:53:44.618409       1 nanny.go:220] dnsmasq exited: <nil>
goroutine 1 [running]:
k8s.io/dns/vendor/github.com/golang/glog.stacks(0xc0002e1500, 0xc000342000, 0x42, 0x95)
	/go/src/k8s.io/dns/vendor/github.com/golang/glog/glog.go:769 +0xd4
k8s.io/dns/vendor/github.com/golang/glog.(*loggingT).output(0x1d686c0, 0xc000000003, 0xc0000da2c0, 0x1cfe8ed, 0x8, 0xdc, 0x0)
	/go/src/k8s.io/dns/vendor/github.com/golang/glog/glog.go:720 +0x329
k8s.io/dns/vendor/github.com/golang/glog.(*loggingT).printf(0x1d686c0, 0x3, 0x127e272, 0x12, 0xc0005bbe08, 0x1, 0x1)
	/go/src/k8s.io/dns/vendor/github.com/golang/glog/glog.go:655 +0x14b
k8s.io/dns/vendor/github.com/golang/glog.Fatalf(0x127e272, 0x12, 0xc0005bbe08, 0x1, 0x1)
	/go/src/k8s.io/dns/vendor/github.com/golang/glog/glog.go:1148 +0x67
k8s.io/dns/pkg/dnsmasq.RunNanny(0x13bc8e0, 0xc00017a1e0, 0x127cd0d, 0x11, 0x1d89108, 0x0, 0x0, 0x0, 0x127af27, 0xf)
	/go/src/k8s.io/dns/pkg/dnsmasq/nanny.go:220 +0x564
main.main()
	/go/src/k8s.io/dns/cmd/dnsmasq-nanny/main.go:82 +0x240
Verifying kube-dns startup
I0414 02:53:46.065150       1 dns.go:48] version: 1.15.11-14-g8f51218
F0414 02:53:46.065298       1 server.go:61] Failed to create a kubernetes client: unable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined
Verifying sidecar startup
I0414 02:53:47.283678       1 main.go:51] Version v1.15.11-14-g8f51218
I0414 02:53:47.283768       1 server.go:46] Starting server (options {DnsMasqPort:53 DnsMasqAddr:127.0.0.1 DnsMasqPollIntervalMs:5000 Probes:[] PrometheusAddr:0.0.0.0 PrometheusPort:10054 PrometheusPath:/metrics PrometheusNamespace:kubedns})
W0414 02:53:47.284807       1 server.go:65] Error getting metrics from dnsmasq: read udp 127.0.0.1:60748->127.0.0.1:53: read: connection refused
W0414 02:53:52.285547       1 server.go:65] Error getting metrics from dnsmasq: read udp 127.0.0.1:41773->127.0.0.1:53: read: connection refused
W0414 02:53:57.286567       1 server.go:65] Error getting metrics from dnsmasq: read udp 127.0.0.1:36680->127.0.0.1:53: read: connection refused
W0414 02:54:02.286998       1 server.go:65] Error getting metrics from dnsmasq: read udp 127.0.0.1:33077->127.0.0.1:53: read: connection refused
W0414 02:54:07.287909       1 server.go:65] Error getting metrics from dnsmasq: read udp 127.0.0.1:59188->127.0.0.1:53: read: connection refused
W0414 02:54:12.288251       1 server.go:65] Error getting metrics from dnsmasq: read udp 127.0.0.1:37611->127.0.0.1:53: read: connection refused
W0414 02:54:17.288888       1 server.go:65] Error getting metrics from dnsmasq: read udp 127.0.0.1:44740->127.0.0.1:53: read: connection refused
W0414 02:54:22.289449       1 server.go:65] Error getting metrics from dnsmasq: read udp 127.0.0.1:34158->127.0.0.1:53: read: connection refused

@champtar
Copy link
Contributor Author

iptables v1.8.2 (nf_tables) => need to switch to legacy !

@champtar
Copy link
Contributor Author

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 14, 2020
@champtar champtar marked this pull request as draft April 14, 2020 03:00
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 14, 2020
@champtar
Copy link
Contributor Author 

$ ./image-checks.sh 1.15.11-14-g423a2b4 champtar
Verifying that iptables exists in node-cache image
iptables v1.8.2 (legacy): no command specified
Try `iptables -h' or 'iptables --help' for more information.
Verifying that node-cache binary exists in node-cache image
2020/04/14 03:18:09 [FATAL] Error parsing flags - Invalid localip specified - "", Exiting
Verifying dnsmasq-nanny startup
E0414 03:18:11.657200       1 nanny.go:154] Error reading from stderr: read |0: file already closed
F0414 03:18:11.662363       1 nanny.go:220] dnsmasq exited: <nil>
goroutine 1 [running]:
k8s.io/dns/vendor/github.com/golang/glog.stacks(0xc0002bb600, 0xc000558000, 0x42, 0x95)
	/go/src/k8s.io/dns/vendor/github.com/golang/glog/glog.go:769 +0xd4
k8s.io/dns/vendor/github.com/golang/glog.(*loggingT).output(0x1d686c0, 0xc000000003, 0xc0000d82c0, 0x1cfe8ed, 0x8, 0xdc, 0x0)
	/go/src/k8s.io/dns/vendor/github.com/golang/glog/glog.go:720 +0x329
k8s.io/dns/vendor/github.com/golang/glog.(*loggingT).printf(0x1d686c0, 0x3, 0x127e272, 0x12, 0xc000629e08, 0x1, 0x1)
	/go/src/k8s.io/dns/vendor/github.com/golang/glog/glog.go:655 +0x14b
k8s.io/dns/vendor/github.com/golang/glog.Fatalf(0x127e272, 0x12, 0xc000629e08, 0x1, 0x1)
	/go/src/k8s.io/dns/vendor/github.com/golang/glog/glog.go:1148 +0x67
k8s.io/dns/pkg/dnsmasq.RunNanny(0x13bc8e0, 0xc0002bb6e0, 0x127cd0d, 0x11, 0x1d89108, 0x0, 0x0, 0x0, 0x127af27, 0xf)
	/go/src/k8s.io/dns/pkg/dnsmasq/nanny.go:220 +0x564
main.main()
	/go/src/k8s.io/dns/cmd/dnsmasq-nanny/main.go:82 +0x240
Verifying kube-dns startup
I0414 03:18:12.978180       1 dns.go:48] version: 1.15.11-14-g423a2b4
F0414 03:18:12.978381       1 server.go:61] Failed to create a kubernetes client: unable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined
Verifying sidecar startup
I0414 03:18:14.234270       1 main.go:51] Version v1.15.11-14-g423a2b4
I0414 03:18:14.234419       1 server.go:46] Starting server (options {DnsMasqPort:53 DnsMasqAddr:127.0.0.1 DnsMasqPollIntervalMs:5000 Probes:[] PrometheusAddr:0.0.0.0 PrometheusPort:10054 PrometheusPath:/metrics PrometheusNamespace:kubedns})
W0414 03:18:14.235816       1 server.go:65] Error getting metrics from dnsmasq: read udp 127.0.0.1:37675->127.0.0.1:53: read: connection refused
W0414 03:18:19.236556       1 server.go:65] Error getting metrics from dnsmasq: read udp 127.0.0.1:35403->127.0.0.1:53: read: connection refused

@champtar
Copy link
Contributor Author

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 14, 2020
@champtar
Copy link
Contributor Author

@prameshj i'll run test champtar/k8s-dns-node-cache-amd64 tomorrow (as it's really needed ...) but I'm not using the other images, can you test them or find people to test them ?
Same question for the other ARCH ?

@prameshj
Copy link
Contributor

@prameshj i'll run test champtar/k8s-dns-node-cache-amd64 tomorrow (as it's really needed ...) but I'm not using the other images, can you test them or find people to test them ?
Same question for the other ARCH ?

Thanks @champtar , let's test all the amd64 images and get help to verify other arch after tagging/promoting the images.

@prameshj
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 14, 2020
@champtar
Copy link
Contributor Author 

# kubectl get pod/nodelocaldns-tv2c4 -n kube-system -o jsonpath='{.spec.containers[*].image}'
champtar/k8s-dns-node-cache-amd64:1.15.11-14-g423a2b4

[root@etienne-ks141 ~]# kubectl logs nodelocaldns-tv2c4 -n kube-system 
2020/04/14 17:55:00 [INFO] Using Corefile /etc/coredns/Corefile
2020/04/14 17:55:00 [ERROR] Failed to read node-cache coreFile /etc/coredns/Corefile.base - open /etc/coredns/Corefile.base: no such file or directory
2020/04/14 17:55:00 [ERROR] Failed to sync kube-dns config directory /etc/kube-dns, err: lstat /etc/kube-dns: no such file or directory
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {raw PREROUTING [-p tcp -d 169.254.25.10 --dport 53 -j NOTRACK]}
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {raw PREROUTING [-p udp -d 169.254.25.10 --dport 53 -j NOTRACK]}
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {filter INPUT [-p tcp -d 169.254.25.10 --dport 53 -j ACCEPT]}
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {filter INPUT [-p udp -d 169.254.25.10 --dport 53 -j ACCEPT]}
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {raw OUTPUT [-p tcp -s 169.254.25.10 --sport 53 -j NOTRACK]}
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {raw OUTPUT [-p udp -s 169.254.25.10 --sport 53 -j NOTRACK]}
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {filter OUTPUT [-p tcp -s 169.254.25.10 --sport 53 -j ACCEPT]}
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {filter OUTPUT [-p udp -s 169.254.25.10 --sport 53 -j ACCEPT]}
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {raw OUTPUT [-p tcp -d 169.254.25.10 --dport 53 -j NOTRACK]}
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {raw OUTPUT [-p udp -d 169.254.25.10 --dport 53 -j NOTRACK]}
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {raw OUTPUT [-p tcp -d 169.254.25.10 --dport 8080 -j NOTRACK]}
2020/04/14 17:55:00 [INFO] Added back nodelocaldns rule - {raw OUTPUT [-p tcp -s 169.254.25.10 --sport 8080 -j NOTRACK]}
2020/04/14 17:55:00 [INFO] Added interface - nodelocaldns
cluster.local.:53 on 169.254.25.10
in-addr.arpa.:53 on 169.254.25.10
ip6.arpa.:53 on 169.254.25.10
.:53 on 169.254.25.10
[INFO] plugin/reload: Running configuration MD5 = 560762611148ea0ade87f4b0320dae04
CoreDNS-1.6.7
linux/amd64, go1.11.13, 
[ERROR] plugin/errors: 2 4780252023805329001.7967571612769004106.in-addr.arpa. HINFO: dial tcp 100.64.0.3:53: connect: no route to host
[ERROR] plugin/errors: 2 8728799070124543715.3143575536389865816.cluster.local. HINFO: dial tcp 100.64.0.3:53: connect: no route to host
[ERROR] plugin/errors: 2 1332211314622154950.8571438893655711917.ip6.arpa. HINFO: dial tcp 100.64.0.3:53: i/o timeout

[root@etienne-ks141 ~]# iptables-save | grep 169.254.25.10
-A PREROUTING -d 169.254.25.10/32 -p udp -m udp --dport 53 -j NOTRACK
-A PREROUTING -d 169.254.25.10/32 -p tcp -m tcp --dport 53 -j NOTRACK
-A OUTPUT -s 169.254.25.10/32 -p tcp -m tcp --sport 8080 -j NOTRACK
-A OUTPUT -d 169.254.25.10/32 -p tcp -m tcp --dport 8080 -j NOTRACK
-A OUTPUT -d 169.254.25.10/32 -p udp -m udp --dport 53 -j NOTRACK
-A OUTPUT -d 169.254.25.10/32 -p tcp -m tcp --dport 53 -j NOTRACK
-A OUTPUT -s 169.254.25.10/32 -p udp -m udp --sport 53 -j NOTRACK
-A OUTPUT -s 169.254.25.10/32 -p tcp -m tcp --sport 53 -j NOTRACK
-A INPUT -d 169.254.25.10/32 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -d 169.254.25.10/32 -p tcp -m tcp --dport 53 -j ACCEPT
-A OUTPUT -s 169.254.25.10/32 -p udp -m udp --sport 53 -j ACCEPT
-A OUTPUT -s 169.254.25.10/32 -p tcp -m tcp --sport 53 -j ACCEPT

@champtar champtar marked this pull request as ready for review April 14, 2020 18:26
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 14, 2020
@champtar
Copy link
Contributor Author

@prameshj champtar/k8s-dns-node-cache-amd64:1.15.11-14-g423a2b4 is now tested and seems fine to me

@champtar
Use debian-base-$(ARCH):v2.0.0 as base image
bdeeba5 
Use --no-install-recommends to not install nftables as it depends on bash
(https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956655)

Use iptables-legacy only until https://bugzilla.netfilter.org/show_bug.cgi?id=1422 is resolved
Once fixed we will switch to k8s.gcr.io/debian-iptables-$(ARCH) to choose iptables-legacy or iptables-nft at run time
@prameshj
Copy link
Contributor

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 15, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: champtar, prameshj

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 15, 2020
@k8s-ci-robot k8s-ci-robot merged commit 559ef88 into kubernetes:master Apr 15, 2020
@champtar champtar deleted the basev2 branch April 15, 2020 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@prameshj prameshj prameshj left review comments

@bowei bowei Awaiting requested review from bowei

Assignees

@prameshj prameshj

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@champtar @k8s-ci-robot @prameshj