add dynamic weight by available replicas

[Garrybest]

Signed-off-by: Garrybest [email protected]

What type of PR is this?
/kind api-change
/kind feature

What this PR does / why we need it:
According to #730, I think we could add a new division preference instead of mixing with Weighted preference. ReplicaDivisionPreferenceDispersal divides replicas into clusters as many as possible, while respecting clusters' resource availabilities during the division.

Which issue(s) this PR fixes:
Fixes #730

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

ReplicaDivisionPreferenceDispersal divides replicas into clusters as many as possible, while respecting clusters' resource availabilities during the division.

[Garrybest]

/cc @kevin-wangzefeng @RainbowMango @qianjun1993 @mrlihanbo @XiShanYongYe-Chang @gf457832386

[karmada-bot]

@Garrybest: GitHub didn't allow me to request PR reviews from the following users: gf457832386.

Note that only karmada-io members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @kevin-wangzefeng @RainbowMango @qianjun1993 @mrlihanbo @XiShanYongYe-Chang @gf457832386

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[RainbowMango]

/assign

[RainbowMango]

Should we describe the algorithm a little bit here, about how to divide the replicas?

[gf457832386]

Hi, if we divide replicas only by MaxAvailableReplicas, it may cause that a cluster is running out of memory but has a lot of CPU left. Do we need to think about solving the problem of "resource fragmentation". ( I didn't express it clearly in last meeting...emmm @RainbowMango

[Garrybest]

As @gf457832386 said, divide replica by MaxAvailableReplicas. For example, when the scheduler tries to schedule workload X in Dispersal mode:

Cluster A max available replica: 6
Cluster B max available replica: 12
Cluster C max available replica: 18

So we could divide the replica by 6:12:18, as 2 of cluster A, 4 of cluster B and 6 of cluster C. It is obvious that the division type has a benefit with cluster load balance.

[Garrybest]

It would be better to add some documents about all scheduling algorithms and user guide. Besides, more comments could be added first. Let me make a change.

[Garrybest]

Does this PR need more discussion? I think we already had a deal on last meeting. But I'm not sure.

[RainbowMango]

@gf457832386 Do you have any comments?

[RainbowMango]

/lgtm
/assign @kevin-wangzefeng
Let's invite Kevin to take a look.

[RainbowMango]

At the last meeting, @kevin-wangzefeng also mentioned another solution that putting the preference into weight list as a dynamic weight list(ClusterPreferences reserved the space for dynamic weight list).

I tried to work out this solution based on @Garrybest's work, and let's see which solution is better.

// ClusterPreferences describes weight for each cluster or for each group of cluster.
type ClusterPreferences struct {
	// StaticWeightList defines the static cluster weight.
	// +optional
	StaticWeightList []StaticClusterWeight `json:"staticWeightList,omitempty"`
	// DynamicWeight specifies the factor to generates dynamic weight list.
	// +optional
	DynamicWeight DynamicWeightFactor `json:"dynamicWeight,omitempty"`
}

// DynamicWeightFactor represents the weight factor.
// For now only support 'AvailableResource', more factors could be extended if there is a need.
type DynamicWeightFactor string

const (
	// DynamicWeightByAvailableResource represents the cluster weight list should be generated according to
	// available resource(available replicas). e.g.
	// The scheduler has selected 3 clusters(A/B/C) and should divide 12 replicas to them.
	//
	// Workload:
	//   Desired replica: 12
	//
	// Cluster:
	//   A:
	//     Max available replica: 6
	//   B:
	//     Max available replica: 12
	//   C:
	//     Max available replica: 18
	//
	// The weight of cluster A:B:C will be 6:12:18(equals to 1:2:3). At last, the assignment would be 'A: 2, B: 4, C: 6'.
	DynamicWeightByAvailableResource DynamicWeightFactor = "AvailableResource"
)

[Garrybest]

Wow, it's nice. What about AvailableReplica instead of AvailableResource? I think the weight is based on available replicas exactly.

[gf457832386]

At the last meeting, @kevin-wangzefeng also mentioned another solution that putting the preference into weight list as a dynamic weight list(ClusterPreferences reserved the space for dynamic weight list).

I tried to work out this solution based on @Garrybest's work, and let's see which solution is better.

// ClusterPreferences describes weight for each cluster or for each group of cluster.
type ClusterPreferences struct {
	// StaticWeightList defines the static cluster weight.
	// +optional
	StaticWeightList []StaticClusterWeight `json:"staticWeightList,omitempty"`
	// DynamicWeight specifies the factor to generates dynamic weight list.
	// +optional
	DynamicWeight DynamicWeightFactor `json:"dynamicWeight,omitempty"`
}

// DynamicWeightFactor represents the weight factor.
// For now only support 'AvailableResource', more factors could be extended if there is a need.
type DynamicWeightFactor string

const (
	// DynamicWeightByAvailableResource represents the cluster weight list should be generated according to
	// available resource(available replicas). e.g.
	// The scheduler has selected 3 clusters(A/B/C) and should divide 12 replicas to them.
	//
	// Workload:
	//   Desired replica: 12
	//
	// Cluster:
	//   A:
	//     Max available replica: 6
	//   B:
	//     Max available replica: 12
	//   C:
	//     Max available replica: 18
	//
	// The weight of cluster A:B:C will be 6:12:18(equals to 1:2:3). At last, the assignment would be 'A: 2, B: 4, C: 6'.
	DynamicWeightByAvailableResource DynamicWeightFactor = "AvailableResource"
)

A better idea!

[kevin-wangzefeng]

/lgtm
/approve

[karmada-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kevin-wangzefeng

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [kevin-wangzefeng]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[RainbowMango]

Congrats!