Increase the maximum number of Security Group rules the controller can create

[dickeyf]

Is your feature request related to a problem?
The Controller needs to create many rules in the Security Group of the Worker Node's ENIs when it create an instance-type NLB or ip-type NLB.

The instance type NLB requires considerably more rules as the backend ports are NodePort which are unique, unlike ip-type back port that tends to have standard values such as 80 or 443.

Describe the solution you'd like
The Security Groups have a maximum number of rules of 60 by default, which can be increased to 200 easily. Increasing this limit beyond 200 requires the decrease of another limit: security group per ENI which is 5 by default.
Increasing the limit to 200 will help, but it is still not enough.

The solution I'd like is that the Controller would add additional Security groups to the worker nodes when there aren't space anymore in the current Security Groups. This would allow increasing the limit to 1000 by spanning these rules over the max number of 5 SGs per ENI.

Describe alternatives you've considered
Increasing the quota beyond 200, and accepting a lower SG per ENI quota.

Since increasing this quota beyond 200 reduces the number of SG per ENI, it's not ideal, as this might affect other components in the same region/account.

There was also PR #2245 which was considered. It optimizes the rules by merging them into ranges for each unique CIDR source. However this was not ideal from a security standpoint as the optimization allows one LB's rules to overwrite another's.

[kishorj]

@dickeyf, thanks for reporting the issue. We cannot increase SG rule quotas beyond the current AWS limit, however I'm working on options to optimize SG rules for NLB as well.

/kind feature
/assign

[dickeyf]

however I'm working on options to optimize SG rules for NLB as well

Is there an issue for this, or a PR? I'm interested to know what those options are.

[dickeyf]

This issue seems to address the same problem: Optimized security group rules #2118

I understand that once AWS allows assigning Security Group to the NLBs, you will extend the same restricted port ranges behavior to the NLBs.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.