update coredns for security best practice

[Ewocker]

• use non-root user
• remove container privilege
• change core port so that no need cap NET_BIND_SERVICE
• update service kube-dns to use targetPort name to be kube-dns compatible on migration

Note:
During migration when kops is patching service kube-dns, duplicate port number might cause issue that only the first port has the correct targetPort name. In this case kubectl replace needs to be use to resolve the problem. See issue kubernetes/kubernetes#47249

[k8s-ci-robot]

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

---

• If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
• If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
• If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
• Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: [email protected]

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[k8s-ci-robot]

Hi @Ewocker. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Ewocker
To complete the pull request process, please assign kashifsaadat
You can assign the PR to them by writing /assign @kashifsaadat in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[johngmyers]

/ok-to-test

[johngmyers]

/retest

[johngmyers]

I'm not quite sure what to do with this apparent bug in the core/v1 ServiceSpec API. It's not like we can neuter the kube-dns service and put in a kube-dns-kops service instead.

We're execing kubectl apply -f, so we don't seem to have much opportunity to recognize the case and do an update instead of a strategic patch.

[johngmyers]

In researching for the kops office hours, I found that "server side apply" is the Kubernetes-level fix for the broken strategic patch. That is a beta feature right now.

The decision from office hours is that server side apply is too new and risky to take on now. The approach would be to get channels to use delete-and-replace instead of apply.

The question then becomes how to decide to do delete-and-replace. Some options I see:

• Pull the "kubectl apply -f" logic into channels so that we can add code to detect when the patch doesn't apply correctly and then choose an alternate update flow.
• Add metadata to the addon to tell channels to use the alternate flow across a particular version boundary.

We should consider whether an option can also address the immutable field problem.

[rifelpet]

While this isn't directly related, I think one other issue that could benefit from adjusting how Kops applies channels is the removal of resources. If a manifest no longer defines a resource (either during an upgrade or downgrade) kubectl apply -f will not delete it which leaves it lingering and unmanaged in the cluster. kubectl apply has a prune field but I'm not sure if that is sufficient. If the new channel application strategy could handle that as well, that'd be great.

[Ewocker]

While this isn't directly related, I think one other issue that could benefit from adjusting how Kops applies channels is the removal of resources. If a manifest no longer defines a resource (either during an upgrade or downgrade) kubectl apply -f will not delete it which leaves it lingering and unmanaged in the cluster. kubectl apply has a prune field but I'm not sure if that is sufficient. If the new channel application strategy could handle that as well, that'd be great.

@rifelpet In this case while transitioning from kubedns to coredns, would kubedns be removed? Is that what we want, or we want the user to manually remove one of them?

[rifelpet]

The current migration instructions require the user manually remove the KubeDNS deployment. It would be great if Kops could do that automatically, but in the case of KubeDNS/CoreDNS I think knowing when it is safe to remove might add significant complexity. All of the CoreDNS pods need to be Ready, the replica count would need to match, etc. I think that might be too much to strive for with this migration.

[Ewocker]

@johngmyers

I am not sure what other options do we have for alternate flow other then replace (delete then create), but I can see some problems of doing that:

• If using kubectl replace but if an addon add a new resource which was not previously in the manifest, then replace will fail. Similar things will happen for first delete then create. If a resource is removed from new version, deletion will miss the removed resources (which is already a problem in kops).
• Workload can have down time because we need to first delete which might not be acceptable for some addon, ex. dns. (This might be the same in the other approach though since we are doing a single apply using a multi-resource file)

In the original comment where I was able to do this without service down time is to only replace the kube-dns service instead of all. But I am not sure if it is possible to implement this today in kops.

In this case I will assume that we need to live with the fact that there can be a blip in this type of immutable or conflicting update.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[k8s-ci-robot]

@Ewocker: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

@Ewocker: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-kops-verify-cloudformation	18b1e22	link	/test pull-kops-verify-cloudformation
pull-kops-verify-hashes	18b1e22	link	/test pull-kops-verify-hashes
pull-kops-e2e-k8s-containerd	18b1e22	link	/test pull-kops-e2e-k8s-containerd

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

[k8s-ci-robot]

@fejta-bot: Closed this PR.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.