TAS: may report negative values of pods in messages for non-admitted workloads

[mimowo]

What happened:

When a workload using preferred topology annotation cannot be admitted is sometimes reports negative number of pods in the status message of the workload.

What you expected to happen:

negative values of pods are never reported, it does not make sense.

How to reproduce it (as minimally and precisely as possible):

I have a reproducible scenario, but it is yet not simplified enough to be included here. I will simplify and post.

Basically I create multiple jobs using preferred, and workload corresponding to the non-admitted job reports message like:

status:
  conditions:
  - lastTransitionTime: "2024-11-07T13:01:01Z"
    message: 'couldn''t assign flavors to pod set main: insufficient unused quota
      for cpu in flavor tas-flavor, 30 more needed, topology "gke-default" allows
      to fit only -18 out of 40 pod(s)'
    observedGeneration: 1
    reason: Pending
    status: "False"
    type: QuotaReserved

Anything else we need to know?:

It was reported first here: #3211 (comment)

[mimowo]

/assign

[mimowo]

/cc @tenzen-y @hy00nc

[mimowo]

After investigation it turns out that this is not only related to preferred - so I retitled the issue.

The root cause is that the usage of TAS workloads is multiplied by the number of resources, because of this loop which iterates over pairs of (resource, flavor). So it would add the same usage for (cpu, tas-flavor), (memory, tas-flavor), (gpu, tas-flavor).

I proposed a fix #3490