Add documentation for noEncap and hybrid modes, and kube-router #1798

jianjuns · 2021-01-29T01:21:49Z

No description provided.

jianjuns · 2021-01-29T01:32:30Z

docs/noencap-hybrid-modes.md

+to the cloud network routers for the Pod CIDRs of Nodes, and then the cloud
+network is able to route Pod traffic between Nodes. This Route Controller
+functionality is supported by the Cloud Provider implementations of the major
+clouds, including: [AWS](https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/legacy-cloud-providers/aws),


Did not find documentations for how to configure cloud providers. Probably we need more work to figure out how to configure each of them.

For in-tree providers, as far as I know, the --configure-cloud-routes flag for kube-controller-manager needs to be set to true. I am not aware of any other requirement. At the moment, the flag also requires --allocate-node-cidrs, but that's a requirement for Antrea anyway.

I believe the flag will work for Azure, GCP, AWS. I don't know about vSphere though, maybe @dantingl knows.

Yes, I know --configure-cloud-routes, but there can be other requirements depending on the clouds. For example, Azure seems require a separate routing table being created.

I do not want to talk about NSX-T configuration here - there will be lots of details (T1, etc.) to talk. Hopefully Danting can enhance vSphere CP documentation for route configuration, then we can just put a link here.

I see, there is the --cloud-config flag that takes an arbitrary configuration file, which is cloud-specific.

jianjuns · 2021-01-29T01:33:03Z

@antoninbas : like to learn what you think about the current version.

codecov-io · 2021-01-29T02:00:35Z

Codecov Report

❗ No coverage uploaded for pull request base (main@e61cfbc). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1798   +/-   ##
=======================================
  Coverage        ?   42.84%           
=======================================
  Files           ?      108           
  Lines           ?    13597           
  Branches        ?        0           
=======================================
  Hits            ?     5825           
  Misses          ?     7286           
  Partials        ?      486

Flag	Coverage Δ
unit-tests	`42.84% <0.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

antoninbas · 2021-01-29T02:12:01Z

docs/noencap-hybrid-modes.md

+in the same subnet. Thus it requires the Node network allow Pod IP addresses
+being sent out from the Nodes' NICs. This requirement is not supported in all


it requires the Node network to allow Pod IP addresses to be sent out

or

it requires that the Node network allow Pod IP addresses to be sent out

antoninbas · 2021-01-29T02:14:23Z

docs/noencap-hybrid-modes.md

+option is defined in `antrea-agent.conf` of the `antrea` ConfigMap in the
+[Antrea deployment YAML](https://github.com/vmware-tanzu/antrea/blob/main/build/yamls/antrea.yml).
+
+```yaml


maybe we should not include this snippet unless there is a strong reason to, as then we will need to keep it in sync I feel like the instructions above are clear enough.

What you think about:

```yaml trafficEncapMode: noEncap

Or:

```yaml antrea-agent.conf: | ... ... trafficEncapMode: noEncap ... ...

second one looks good to me

antoninbas · 2021-01-29T02:15:36Z

docs/noencap-hybrid-modes.md

+    ... ...
+```
+
+After changing the option, you can deploy Antrea with the `Hybrid` mode by:


Suggested change

After changing the option, you can deploy Antrea with the `Hybrid` mode by:

After changing the configuration option, you can deploy Antrea in `Hybrid` mode with the usual command:

antoninbas · 2021-01-29T02:26:33Z

docs/noencap-hybrid-modes.md

+to the cloud network routers for the Pod CIDRs of Nodes, and then the cloud
+network is able to route Pod traffic between Nodes. This Route Controller
+functionality is supported by the Cloud Provider implementations of the major
+clouds, including: [AWS](https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/legacy-cloud-providers/aws),


For in-tree providers, as far as I know, the --configure-cloud-routes flag for kube-controller-manager needs to be set to true. I am not aware of any other requirement. At the moment, the flag also requires --allocate-node-cidrs, but that's a requirement for Antrea anyway.

I believe the flag will work for Azure, GCP, AWS. I don't know about vSphere though, maybe @dantingl knows.

antoninbas · 2021-01-29T02:31:31Z

docs/noencap-hybrid-modes.md

+options are defined in `antrea-agent.conf` of the `antrea` ConfigMap in the
+[Antrea deployment YAML](https://github.com/vmware-tanzu/antrea/blob/main/build/yamls/antrea.yml).
+
+```yaml


same comment as above

antoninbas · 2021-01-29T02:33:17Z

docs/noencap-hybrid-modes.md

+
+We can run kube-router in the advertisement-only mode to advertise Pod CIDRs to
+the peered routers, so the routers can know how to route Pod traffic to the
+Nodes. To deploy kube-router in the advertisement-only, first download the


To deploy kube-router in advertisement-only mode

antoninbas · 2021-01-29T02:34:06Z

docs/noencap-hybrid-modes.md

+The BGP peers should be configured by specifying the --peer-router-asns and
+--peer-router-ips parameters. Note, the ASNs and IPs must match the


nit: use back ticks around --peer-router-asns and --peer-router-ips

antoninbas · 2021-01-29T02:34:20Z

docs/noencap-hybrid-modes.md

+- "--peer-router-asns=65000,65000"
+```
+
+Then you can deploy the kube-router DaemonSet by:


antoninbas · 2021-01-29T02:35:54Z

docs/noencap-hybrid-modes.md

+Antrea can be deployed either before or after kube-router, with the `NoEncap`
+mode.


s/with the NoEncap mode/in NoEncap mode

antoninbas · 2021-01-29T02:36:10Z

docs/noencap-hybrid-modes.md

+kubectl apply -f generic-kuberouter-only-advertise-routes.yaml
+```
+
+You can verify the kube-router Pods are running on the Nodes of your Kubernetes


verify that the kube-router Pods

antoninbas · 2021-01-29T18:09:16Z

docs/noencap-hybrid-modes.md

+    ... ...
+    trafficEncapMode: noEncap
+
+    #noSNAT: false


I think you mean noSNAT: true?

I wanted to show the default value, but yes it seems confusing. Changed to: noSNAT: false.

antoninbas

LGTM

antoninbas · 2021-01-29T23:35:53Z

docs/noencap-hybrid-modes.md

+    ... ...
+    trafficEncapMode: noEncap
+
+    noSNAT: false


noSNAT: false # or true to disable Antrea SNAT for external traffic

jianjuns · 2021-01-30T04:35:38Z

/skip-all

vmwclabot added the cla-not-required label Jan 29, 2021

jianjuns force-pushed the noencap-doc branch 3 times, most recently from 2e85763 to a682647 Compare January 29, 2021 01:30

jianjuns commented Jan 29, 2021

View reviewed changes

antoninbas reviewed Jan 29, 2021

View reviewed changes

jianjuns force-pushed the noencap-doc branch 2 times, most recently from 82aac67 to e7ea391 Compare January 29, 2021 04:24

antoninbas reviewed Jan 29, 2021

View reviewed changes

jianjuns force-pushed the noencap-doc branch from e7ea391 to 58b6596 Compare January 29, 2021 22:51

Add documentation for noEncap and hybrid modes, and kube-router

c2c5a2b

antoninbas previously approved these changes Jan 29, 2021

View reviewed changes

jianjuns dismissed antoninbas’s stale review via c2c5a2b January 29, 2021 23:37

jianjuns force-pushed the noencap-doc branch from 58b6596 to c2c5a2b Compare January 29, 2021 23:37

antoninbas approved these changes Jan 29, 2021

View reviewed changes

jianjuns merged commit 282c159 into antrea-io:main Jan 30, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add documentation for noEncap and hybrid modes, and kube-router #1798

Add documentation for noEncap and hybrid modes, and kube-router #1798

jianjuns commented Jan 29, 2021

jianjuns Jan 29, 2021

antoninbas Jan 29, 2021

jianjuns Jan 29, 2021

jianjuns Jan 29, 2021

antoninbas Jan 29, 2021

jianjuns commented Jan 29, 2021 •

edited

Loading

codecov-io commented Jan 29, 2021 •

edited

Loading

antoninbas Jan 29, 2021

jianjuns Jan 29, 2021

antoninbas Jan 29, 2021

jianjuns Jan 29, 2021 •

edited

Loading

antoninbas Jan 29, 2021

antoninbas Jan 29, 2021

jianjuns Jan 29, 2021

antoninbas Jan 29, 2021

antoninbas Jan 29, 2021

antoninbas Jan 29, 2021

jianjuns Jan 29, 2021

antoninbas Jan 29, 2021

jianjuns Jan 29, 2021

antoninbas Jan 29, 2021

jianjuns Jan 29, 2021

antoninbas Jan 29, 2021

jianjuns Jan 29, 2021

antoninbas Jan 29, 2021

jianjuns Jan 29, 2021

antoninbas Jan 29, 2021

jianjuns Jan 29, 2021

antoninbas left a comment

antoninbas Jan 29, 2021

jianjuns commented Jan 30, 2021

		in the same subnet. Thus it requires the Node network allow Pod IP addresses
		being sent out from the Nodes' NICs. This requirement is not supported in all

	After changing the option, you can deploy Antrea with the `Hybrid` mode by:
	After changing the configuration option, you can deploy Antrea in `Hybrid` mode with the usual command:

		The BGP peers should be configured by specifying the --peer-router-asns and
		--peer-router-ips parameters. Note, the ASNs and IPs must match the

		Antrea can be deployed either before or after kube-router, with the `NoEncap`
		mode.

Add documentation for noEncap and hybrid modes, and kube-router #1798

Add documentation for noEncap and hybrid modes, and kube-router #1798

Conversation

jianjuns commented Jan 29, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jianjuns commented Jan 29, 2021 • edited Loading

codecov-io commented Jan 29, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jianjuns Jan 29, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antoninbas left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jianjuns commented Jan 30, 2021

jianjuns commented Jan 29, 2021 •

edited

Loading

codecov-io commented Jan 29, 2021 •

edited

Loading

jianjuns Jan 29, 2021 •

edited

Loading