Add cleanup-ipvs flag for kube-proxy

[m1093782566]

What this PR does / why we need it:

There is no way to tell if a given ipvs rule is created by ipvs proxier or not, and some people have complained that iptables/userspace proxier will clean up their ipvs rules when start up - both iptables and userspace proxiers need to clean up legacy proxy rules created by ipvs proxier.

This PR adds a new --cleanup-ipvs flag for kube-proxy for the sake of providing users a way to decide if clean up IPVS rules or not when start iptables or userspace proxier.

Which issue(s) this PR fixes:
Fixes #55857

Special notes for your reviewer:

Release note:

Add cleanup-ipvs flag for kube-proxy 

/sig network
/area ipvs
/king bug

[m1093782566]

/assign @thockin

/cc @pires @caseydavenport @roffe @vfreex

[thockin]

/lgtm
/approve

[fejta-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

[fejta-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

[fejta-bot]

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

[bmcustodio]

@m1093782566 how does this affect --cleanup? There are scenarios in which one wants to cleanup previous iptables rules but do not touch IPVS rules, and from this comment I was under the impression we would get --cleanup=<iptables|ipvs>.

(Sorry to ask but I didn't have the time to look into the PR code.)

[m1093782566]

@brunomcustodio

There are scenarios in which one wants to cleanup previous iptables rules but do not touch IPVS rules

This PR doesn't cover the case you mentioned.

and from this comment I was under the impression we would get --cleanup=<iptables|ipvs>.

I didn't do that because the flag --cleanup means Clean AND Exist. We can't use the flag in iptables/userspace proxier start up.

[m1093782566]

BTW, there is a deprecated flag --cleanup-iptables. If we really want to "cleanup previous iptables rules but do not touch IPVS rules", probably we need to make the flag alive. However, it deserves more discussion and is out of scope for this PR.

[k8s-github-robot]

[MILESTONENOTIFIER] Milestone Pull Request Labels Incomplete

@dchen1107 @m1093782566 @mikedanese @thockin

Action required: This pull request requires label changes. If the required changes are not made within 2 days, the pull request will be moved out of the v1.9 milestone.

kind: Must specify exactly one of kind/bug, kind/cleanup or kind/feature.
priority: Must specify exactly one of priority/critical-urgent, priority/important-longterm or priority/important-soon.

Help

• Additional instructions
• Commands for setting labels

[thockin]

/lgtm

[k8s-github-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: m1093782566, thockin

Associated issue: 55857

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

• cmd/OWNERS [thockin]
• pkg/proxy/OWNERS [thockin]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

[k8s-github-robot]

Automatic merge from submit-queue (batch tested with PRs 55103, 56036, 56186). If you want to cherry-pick this change to another branch, please follow the instructions here.

[cblecker]

@m1093782566 @thockin This change broke cross build on Windows:

I1122 09:21:38.273] # k8s.io/kubernetes/cmd/kube-proxy/app
I1122 09:21:38.273] cmd/kube-proxy/app/server.go:221:36: too many arguments in call to NewProxyServer
I1122 09:21:38.273] 	have (*kubeproxyconfig.KubeProxyConfiguration, bool, bool, *"k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime".Scheme, string)
I1122 09:21:38.273] 	want (*kubeproxyconfig.KubeProxyConfiguration, bool, *"k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/runtime".Scheme, string)

kubernetes/cmd/kube-proxy/app/server_windows.go

Line 49 in 4e2f5e2

func NewProxyServer(config *proxyconfigapi.KubeProxyConfiguration, cleanupAndExit bool, scheme *runtime.Scheme, master string) (*ProxyServer, error) {

[Manikantayalagandala]

Hi Team ,

Any final cmd to resolve the issue.