pkg/asset: calico: Enable hostPort for calico networking & update cniVersion to 0.3.1 #711

weikinhuang · 2017-09-15T01:00:51Z

No description provided.

coreosbot · 2017-09-15T01:00:53Z

Can one of the admins verify this patch?

coreosbot · 2017-09-15T01:00:53Z

Can one of the admins verify this patch?

coreosbot · 2017-09-15T01:00:53Z

Can one of the admins verify this patch?

coreosbot · 2017-09-15T01:00:53Z

Can one of the admins verify this patch?

coreosbot · 2017-09-15T01:00:53Z

Can one of the admins verify this patch?

coreosbot · 2017-09-15T01:00:53Z

Can one of the admins verify this patch?

k8s-ci-robot · 2017-09-15T01:00:53Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://github.com/kubernetes/kubernetes/wiki/CLA-FAQ to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
If you have done the above and are still having issues with the CLA being reported as unsigned, please email the CNCF helpdesk: [email protected]

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

weikinhuang · 2017-09-15T01:05:30Z

I have tested on my local cluster bootstrapped with bootkube and the --experimental-calico-network-policy flag. Per the comments in #698 and #697 this configuration won't be loaded by default on an existing cluster running calico because 10-calico.conf will be loaded ahead of 10-calico.conflist.

This should work out of the box for new clusters. For existing clusters 10-calico.conf must be deleted on every node and the daemonset must be restarted.

weikinhuang · 2017-09-15T03:19:09Z

I signed the CLA

squeed · 2017-09-19T16:55:00Z

pkg/asset/internal/templates.go

-        "nodename": "__KUBERNETES_NODE_NAME__",
-        "ipam": {
+      "name": "k8s-pod-network",
+      "cniVersion": "0.3.0",


This should be 0.3.1 - we fixed a small bug. It doesn't require any other changes.

squeed · 2017-09-19T16:55:54Z

FWIW, the changes in here will also work with Flannel. I can file a PR to update that.

dghubble · 2017-09-19T17:33:42Z

@squeed this change is made to flannel in #697, but paired with changes in the flannel-cni image to cleanup the old CNI config in a way that is backwards compatible whether folks want to fix hostPort or not.

weikinhuang · 2017-09-19T21:42:55Z

Updated cniVersion

dghubble · 2017-09-28T23:39:21Z

bootkube won't properly support Calico networking until #714. Installing policy-only Calico on top of cluster using flannel connectivity, this may work because the new CNI config is ordered first. But its a hack on top of a hack.

Let's test this following #714, in a setup that does not use flannel at all and runs Calico networking alone.

dghubble · 2017-10-02T21:33:24Z

Rebase please

weikinhuang · 2017-10-02T21:39:32Z

done

dghubble · 2017-10-10T06:13:28Z

coreosbot run e2e

redbaron · 2017-12-06T15:12:10Z

pkg/asset/internal/templates.go

+        {
+          "type": "portmap",
+          "capabilities": {
+            "portMappings": true


adding

"externalSetMarkChain": "KUBE-MARK-MASQ"

will reuse existing iptables chains

Not yet - we have to do a plugins release first :-)

latest bootkube uses https://github.com/projectcalico/cni-plugin/releases/tag/v1.11.1 which already includes portmap plugin. Isn't it enough?

Pending support for CNI_OLD_CONF_NAME

Do we need to wait for it?

is bootkube concerned with compatibility with existing clusters?

Should you name it 05-calico.conflist it will be loaded ahead of existing 10-calico.conf as per https://github.com/kubernetes/kubernetes/blob/v1.8.4/pkg/kubelet/network/cni/cni.go#L107

redbaron · 2017-12-06T15:14:14Z

Successfully bootstrapped clusters with bootkube 0.9, this change and calico

dghubble · 2017-12-11T18:26:07Z

We're also interested in this, are we waiting for the 3.x version release?

Yes.

Do we need to wait for it?

We waited for flannel-cni to provide a mechanism to rotate between CNI configs (and cleanup the old manifest) to make migrations easier for downstreams. Its true you can name a new config to be alphabetically first and cleanup the old config manually or later (not great). Depends how long we'd like to wait.

redbaron · 2017-12-12T08:05:11Z

We waited for flannel-cni

TBH I fail to see what value flannel adds comparing to calico mode where it does ipip tunneling. vxlan gives L2 overlay network, but 99.999% of users don't need L2 in kube

squeed · 2017-12-12T11:29:38Z

Flannel doesn't need to rotate configs, only Kubelet does - and it will always use the "first" CNI configuration file, for which it scans every 5 seconds. The challenge, then, is ensuring that a network created with config A can be deleted with config B.

In this case, it will work, since CNI delete is idempotent, and the only change is adding the Portmap plugin.

redbaron · 2017-12-12T15:12:04Z

@squeed , OMG, you say, that kubelet doesn't "memoize' cni config for each network it created so that it can reuse it at tear down time? That is unfortunate oversight.

dghubble · 2017-12-21T16:58:29Z

Looks like config cleanup is now in https://github.com/projectcalico/calico/releases/tag/v2.6.4

weikinhuang · 2017-12-21T17:00:55Z

@dghubble should I update the version number, or do you want to do that in a separate PR, and I'll rebase this on top after a merge?

dghubble · 2017-12-21T17:48:21Z

Let's keep the version bump separate for revertability. Kicked off e2e tests in #818

dghubble · 2017-12-21T19:12:56Z

Gah, the separate sidecar image didn't get bumped to v1.11.1

weikinhuang · 2017-12-21T20:24:14Z

haha, I'll rebase after #819 is merged

weikinhuang · 2018-01-02T19:45:52Z

This has been rebased against #819

weikinhuang · 2018-01-03T18:31:50Z

ping @dghubble

dghubble · 2018-01-04T05:58:24Z

Looks fine, but I'm on vacation til Monday and haven't validated the in-place conf edit/migration yet. We also should be sure to have a changelog note, similar to the one for flannel a few cycles back. Ping @diegs

…

On Wed, Jan 3, 2018, 7:31 PM Wei Kin Huang ***@***.***> wrote: ping @dghubble <https://github.com/dghubble> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#711 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACJidJ_rdi8u2Rwd5AGCfQCyfjTcHX6Iks5tG8eYgaJpZM4PYXMe> .

diegs · 2018-01-04T18:56:29Z

ok to test

dghubble · 2018-01-06T21:36:27Z

pkg/asset/internal/templates.go

+            - name: CNI_CONF_NAME
+              value: 10-calico.conflist
+            - name: CNI_OLD_CONF_NAME
+              value: 10-calico.conf


I see this is the default in the sidecar so it could be left off.

* Ask the Calico sidecar to add a CNI conflist to each node (for calico and portmap plugins). Cleans up Switch from CNI conf to conflist * https://github.com/projectcalico/cni-plugin/blob/v1.11.2/k8s-install/scripts/install-cni.sh * Related kubernetes-retired/bootkube#711

dghubble

I've been running this and its been good. 👍

rel. poseidon/terraform-render-bootstrap#36

dghubble · 2018-01-08T15:06:52Z

Release notes blurb:

Change the default Calico CNI config to enable portmap to fix hostPort (@weikinhuang pkg/asset: calico: Enable hostPort for calico networking & update cniVersion to 0.3.1 #711 )
- Sidecar will automatically cleanup any old CNI 10-calico.conf
- To retain the old behavior, set CNI_CONF_NAME to 10-calico.conf and continue using the old calico-config ConfigMap (i.e. lacking the portmap plugin configuration)

* node_exporter service endpoints run on hostNetwork port 9100 * Re-evaluate after kubernetes-retired/bootkube#711

k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Sep 15, 2017

weikinhuang force-pushed the calico-hostport branch 5 times, most recently from 53ee478 to 0094350 Compare September 15, 2017 01:18

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Sep 15, 2017

squeed reviewed Sep 19, 2017

View reviewed changes

weikinhuang force-pushed the calico-hostport branch 2 times, most recently from fa9e3a3 to 9f7f143 Compare September 19, 2017 18:08

kubernetes-retired deleted a comment from coreosbot Sep 28, 2017

weikinhuang force-pushed the calico-hostport branch from 9f7f143 to b8fa79b Compare October 2, 2017 21:38

redbaron reviewed Dec 6, 2017

View reviewed changes

dghubble mentioned this pull request Dec 21, 2017

pkg/asset: Update Calico from v2.6.3 to v2.6.4 #818

Merged

weikinhuang force-pushed the calico-hostport branch from 1c1ddf1 to 5964dfe Compare December 21, 2017 18:38

weikinhuang force-pushed the calico-hostport branch from 5964dfe to bb32bb8 Compare January 2, 2018 19:45

dghubble reviewed Jan 6, 2018

View reviewed changes

dghubble mentioned this pull request Jan 6, 2018

Calico bump, update cniVersion, and enable portmap to fix hostPort poseidon/terraform-render-bootstrap#36

Merged

weikinhuang added 2 commits January 6, 2018 17:11

Enable portmap for calico

7a7f918

Upgrade calico config cniVersion to 0.3.1

b2884c6

weikinhuang force-pushed the calico-hostport branch from bb32bb8 to b2884c6 Compare January 6, 2018 22:12

dghubble self-requested a review January 8, 2018 15:00

dghubble approved these changes Jan 8, 2018

View reviewed changes

dghubble merged commit 4e38084 into kubernetes-retired:master Jan 8, 2018

justaugustus pushed a commit to justaugustus/typhoon-azure that referenced this pull request Feb 12, 2018

Add firewall rules to allow prometheus to reach node-exporter

77f1ab8

* node_exporter service endpoints run on hostNetwork port 9100 * Re-evaluate after kubernetes-retired/bootkube#711

cknowles mentioned this pull request Jul 15, 2018

Tweaks to network config kubernetes-retired/kube-aws#1407

Closed

pkg/asset: calico: Enable hostPort for calico networking & update cniVersion to 0.3.1 #711

pkg/asset: calico: Enable hostPort for calico networking & update cniVersion to 0.3.1 #711

Conversation

weikinhuang commented Sep 15, 2017

coreosbot commented Sep 15, 2017

coreosbot commented Sep 15, 2017

coreosbot commented Sep 15, 2017

coreosbot commented Sep 15, 2017

coreosbot commented Sep 15, 2017

coreosbot commented Sep 15, 2017

k8s-ci-robot commented Sep 15, 2017

weikinhuang commented Sep 15, 2017

weikinhuang commented Sep 15, 2017

squeed Sep 19, 2017 • edited Loading

Choose a reason for hiding this comment

weikinhuang Sep 19, 2017

Choose a reason for hiding this comment

squeed commented Sep 19, 2017

dghubble commented Sep 19, 2017

weikinhuang commented Sep 19, 2017

dghubble commented Sep 28, 2017

dghubble commented Oct 2, 2017

weikinhuang commented Oct 2, 2017

dghubble commented Oct 10, 2017

redbaron Dec 6, 2017

Choose a reason for hiding this comment

squeed Dec 6, 2017

Choose a reason for hiding this comment

redbaron Dec 6, 2017

Choose a reason for hiding this comment

weikinhuang Dec 6, 2017

Choose a reason for hiding this comment

redbaron Dec 6, 2017

Choose a reason for hiding this comment

redbaron commented Dec 6, 2017

dghubble commented Dec 11, 2017

redbaron commented Dec 12, 2017 • edited Loading

squeed commented Dec 12, 2017

redbaron commented Dec 12, 2017

dghubble commented Dec 21, 2017

weikinhuang commented Dec 21, 2017

dghubble commented Dec 21, 2017 • edited Loading

dghubble commented Dec 21, 2017

weikinhuang commented Dec 21, 2017

weikinhuang commented Jan 2, 2018

weikinhuang commented Jan 3, 2018

dghubble commented Jan 4, 2018 via email

diegs commented Jan 4, 2018

dghubble Jan 6, 2018

Choose a reason for hiding this comment

dghubble left a comment

Choose a reason for hiding this comment

dghubble commented Jan 8, 2018

squeed Sep 19, 2017 •

edited

Loading

redbaron commented Dec 12, 2017 •

edited

Loading

dghubble commented Dec 21, 2017 •

edited

Loading