Calico enable support for eBPF #7618

cristicalin · 2021-05-17T18:00:02Z

What type of PR is this?
/kind feature

What this PR does / why we need it:
Calico supports the eBPF dataplane in more modern distributions like Ubuntu 20.04, CentOS 8 (and streams) and newer. This PR enables the necessary capabilities to enable eBPF.

While this PR does not provide a single setting (due to the nature of how eBPF is enabled in Calico), it does document the complete process to set it up.

For the time being, we are missing way for calico-node to access the kube-api correctly. The current assumption in Calico is that there is an external loadbalancer that services the kubernetes cluster which may not be true for some kubespray deployments.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Calico: Support enabling the eBPF dataplane for Calico

k8s-ci-robot · 2021-05-17T18:00:10Z

Hi @cristicalin. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

oomichi

/ok-to-test

oomichi · 2021-05-18T17:24:48Z

roles/network_plugin/calico/tasks/check.yml

+- name: "Check supported calico versions"
+  assert:
+    that:
+     - "calico_version in calico_crds_archive_checksums.keys()"


Here we need to fix indent to pass yamlint job.

$ yamllint --strict . ./roles/network_plugin/calico/tasks/check.yml 5:6 error wrong indentation: expected 6 but found 5 (indentation)

Sorry about that, it is fixed in the last push.

oomichi

Thanks for updating, still some questions before merging.

/ok-to-test

oomichi · 2021-05-19T22:48:32Z

roles/network_plugin/calico/tasks/check.yml

+  assert:
+    that:
+      - "calico_version is version('v3.19.0','>=')"
+    msg: "you should use calico 3.19.0 or newer with eBPF dataplane"


Just question:
According to https://www.projectcalico.org/introducing-the-calico-ebpf-dataplane/ which is mentioned in this pr, eBPF dataplane is implemented since Calico v3.13 as tech review.
Is that GA since v3.19?

No, this is just from some discussions I had with project calico staff on their slack. I actually did the development for this PR on 3.18.3 and it worked OK but their recommendation was to use 3.19.0 as they would not invest into debugging older releases.

We can relax this completely but I would need to test with all the older versions to confirm behavior.

I've tested 3.16.10, 3.17.4 and 3.18.3 and I was able to do successful deployments with all 3 versions. As such I removed this dependency and dropped the 3.19.0 change from this PR in favor of opening a separate one for the new calico version: #7630

oomichi · 2021-05-19T22:49:32Z

roles/kubernetes-apps/policy_controller/calico/templates/calico-kube-controllers.yml.j2

+            periodSeconds: 10
+            initialDelaySeconds: 10
+            failureThreshold: 6
+{% endif %}


Is the above change for Calico v3.19.0? not for eBPF?
I could not find the relationship between the above change and eBPF.

This is for v3.19.0 not strictly for eBPF. This liveness probe mechanism is not available in 3.18 and prior versions.

@cristicalin Thanks for your clear explanation, I got it.
Then it is better to merge #7630 before this.

oomichi · 2021-05-19T22:51:45Z

roles/network_plugin/calico/templates/calico-cr.yml.j2

@@ -39,8 +42,6 @@ rules:
    verbs:
    - use
 {% elif calico_datastore == "kdd" %}
-      # Calico stores some configuration information in node annotations.
-      - update


Why do we need to move this part for eBPF?

This is actually an unrelated fix, if you look at the upstream manifests the update applies to the nodes/status resource not policy. I can't help feeling this was actually a security issue in kubespray deployments allowing Calico service account to update policies that it shouldn't.

So, I did some re-reading of this portion of the code and it seems that node annotations are only relevant with kdd in which case I was wrong to move that part around, I fixed it in the new push.

cristicalin · 2021-05-21T06:22:23Z

/retest

oomichi · 2021-05-26T16:56:41Z

/lgtm

* allow enabling typha prometheus metrics

* manage the kubernetes-services-endpoint configmap

* enforce disabling kube-proxy when using eBPF dataplane * ensure calico_version is supported

cristicalin · 2021-06-01T15:19:13Z

/retest

oomichi · 2021-06-01T20:34:19Z

/lgtm

k8s-ci-robot · 2021-06-07T11:55:39Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cristicalin, floryut

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [floryut]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* Calico: align manifests with upstream * allow enabling typha prometheus metrics * Calico: enable eBPF support * manage the kubernetes-services-endpoint configmap * Calico: document the use of eBPF dataplane * Calico: improve checks before deployment * enforce disabling kube-proxy when using eBPF dataplane * ensure calico_version is supported

k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 17, 2021

k8s-ci-robot requested review from EppO and oomichi May 17, 2021 18:00

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 17, 2021

cristicalin marked this pull request as draft May 18, 2021 07:55

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 18, 2021

cristicalin force-pushed the calico_manifests_alignment branch 3 times, most recently from 470fbbe to 65b1798 Compare May 18, 2021 09:33

cristicalin marked this pull request as ready for review May 18, 2021 09:39

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 18, 2021

oomichi reviewed May 18, 2021

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 18, 2021

cristicalin force-pushed the calico_manifests_alignment branch 2 times, most recently from 1c0f632 to a2439c7 Compare May 18, 2021 20:10

oomichi reviewed May 19, 2021

View reviewed changes

cristicalin force-pushed the calico_manifests_alignment branch 2 times, most recently from a3657c3 to 99b2cd7 Compare May 20, 2021 10:44

cristicalin mentioned this pull request May 21, 2021

Add Calico Wireguard support #7637

Closed

cristicalin force-pushed the calico_manifests_alignment branch from 99b2cd7 to 2c9cf58 Compare May 21, 2021 09:47

cristicalin mentioned this pull request May 21, 2021

Calico wireguard #7638

Merged

cristicalin force-pushed the calico_manifests_alignment branch from 2c9cf58 to 76e22ea Compare May 21, 2021 11:23

k8s-ci-robot assigned oomichi May 26, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 26, 2021

cristicalin added 4 commits May 30, 2021 14:07

Calico: align manifests with upstream

9267f82

* allow enabling typha prometheus metrics

Calico: enable eBPF support

958d150

* manage the kubernetes-services-endpoint configmap

Calico: document the use of eBPF dataplane

d0d69f4

Calico: improve checks before deployment

5f43fb1

* enforce disabling kube-proxy when using eBPF dataplane * ensure calico_version is supported

cristicalin force-pushed the calico_manifests_alignment branch from 76e22ea to 5f43fb1 Compare May 30, 2021 14:13

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 30, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 1, 2021

floryut approved these changes Jun 7, 2021

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 7, 2021

k8s-ci-robot merged commit ec0c0d4 into kubernetes-sigs:master Jun 7, 2021

floryut mentioned this pull request Sep 8, 2021

Release Proposal v2.17 #7951

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Calico enable support for eBPF #7618

Calico enable support for eBPF #7618

cristicalin commented May 17, 2021 •

edited

Loading

k8s-ci-robot commented May 17, 2021

oomichi left a comment

oomichi May 18, 2021

cristicalin May 18, 2021

oomichi left a comment

oomichi May 19, 2021

cristicalin May 20, 2021

cristicalin May 20, 2021

oomichi May 19, 2021

cristicalin May 20, 2021

oomichi May 20, 2021

oomichi May 19, 2021

cristicalin May 20, 2021

cristicalin May 20, 2021

cristicalin commented May 21, 2021

oomichi commented May 26, 2021

cristicalin commented Jun 1, 2021

oomichi commented Jun 1, 2021

k8s-ci-robot commented Jun 7, 2021

Calico enable support for eBPF #7618

Calico enable support for eBPF #7618

Conversation

cristicalin commented May 17, 2021 • edited Loading

k8s-ci-robot commented May 17, 2021

oomichi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oomichi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cristicalin commented May 21, 2021

oomichi commented May 26, 2021

cristicalin commented Jun 1, 2021

oomichi commented Jun 1, 2021

k8s-ci-robot commented Jun 7, 2021

cristicalin commented May 17, 2021 •

edited

Loading