Kubernetes networking over AWS VPC

[camilb]

https://github.com/aws/amazon-vpc-cni-k8s

[mumoshu]

Fun info: Calico will be enchanced to utilize Security Groups as the backend for network policies
Ref: aws/amazon-vpc-cni-k8s#7 (comment)

[mumoshu]

I'm really looking forward to this!

My only concern is that the limit of IPs per ENI e.g. 10 for c4.large. 10 pods per node at maximum? Sounds a bit low.

It is indeed better than the cni plugin for ECS which is limited by the number of max ENIs per EC2 instance(3 for c3.large. Only 2 pods per node?), though.

Ref: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI

[liwenwu-amazon]

A c4.large node should be able to run 27 Pods.
Here is the calculation:
3 ENIs * (10 IP addresses -1)

[mumoshu]

@liwenwu-amazon Thanks for chiming in!

Yes, it should -
Does amazon-vpc-cni-k8s already support utilizing multiple ENIs per EC2 instance to serve e.g. 3 ENIs * (10 IPs - 1), btw?

[liwenwu-amazon]

Yes.

[pawelprazak]

So if I understand it correctly, according to the proposal the calculation would be e.g.:

Max IPs = min((N * M - N), subnet's free IP)

for m4.xlarge: 4*15-4 = 56 pods max

please correct me if I'm wrong

[camilb]

@mumoshu This looks interesting: https://github.com/lyft/cni-ipvlan-vpc-k8s

https://eng.lyft.com/announcing-cni-ipvlan-vpc-k8s-ipvlan-overlay-free-kubernetes-networking-in-aws-95191201476e

[mumoshu]

@liwenwu-amazon Great! Thanks for clarifying 👍

Would you also mind enlightening us about how amazon-vpc-cni-k8s compares with cni-ipvlan-vpc-k8s?

One gotcha of the latter from lyft folks is that source pod IPs are lost in the Pod-Svc-Pod communication, like kube-proxy was in the user-space mode.

amazon-vpc-cni-k8s doesn't have a such restriction, right?

[mumoshu]

https://twitter.com/paulnivin/status/937877736369954816

[liwenwu-amazon]

amazon-vpc-cni-k8s will NOT change pod IPs for Pod-Svc-Pod communication.

[mumoshu]

@liwenwu-amazon Thanks again for the clarification.
Really looking forward to try it - Please keep up the great work!

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.