DRA: structured parameters

[pohly]

Enhancement Description

• One-line enhancement description (can be used as a release note): The original dynamic resource allocation (DRA) uses claim and class parameters that are opaque to Kubernetes. With structured parameters, kube-scheduler and Cluster Autoscaler can handle respectively simulate claim allocation themselves without relying on a third-party driver.

• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/4381-dra-structured-parameters/

• Discussion Link: Batch WG meeting 20231207, DRA intro and plan in that meeting

• Primary contact (assignee): @pohly

• Responsible SIGs: SIG Node

• Enhancement target (which target equals to which milestone):

  • Alpha release target (x.y): 1.30-1.31
  • Beta release target (x.y): 1.32
  • Stable release target (x.y):

• Alpha in 1.30

  • KEP (k/enhancements) update PR(s): KEP 4381: add structured parameters for dynamic resource allocation #4384
  • Code (k/k) update PR(s): DRA: structured parameters kubernetes#123516
  • Docs (k/website) update PR(s): DRA in Kubernetes 1.30: adds structured parameters website#45287

• Alpha in 1.31

  • KEP (k/enhancements) update PR(s): KEP-4381: DRA update for 1.31 #4709
  • Code (k/k) update PR(s): DRA for 1.31 kubernetes#125488
  • Docs (k/website) update PR(s): DRA documentation for 1.31 website#46816

• Beta in 1.32

  • KEP (k/enhancements) update PR(s): KEP-4381: DRA: PRR and API for beta of structured parameters in 1.32 #4869
  • Code (k/k) update PR(s):
    • DRA resourceclaims: maintain metric of total and allocated claims kubernetes#127661
    • DRA API: AdminAccess in DeviceRequestAllocationResult + DRAAdminAccess feature gate kubernetes#127266
    • DRA API: implement CEL cost limit kubernetes#128101
    • Kubelet: add DRA latency metrics kubernetes#127146
    • DRA resourceclaims: maintain metric of total and allocated claims kubernetes#127661
    • DRA 1.32 API: promotion to beta kubernetes#127511
  • Docs (k/website) update(s): DRA 1.32: beta website#48330

[pohly]

/sig node
/wg batch

The rationale behind creating a separate KEP is that this is an extension of DRA that (at least in theory) could graduate at its own pace. The KEP document itself also will be shorter, which will make discussing it easier.

[SergeyKanzhelev]

/stage alpha
/milestone v1.30

[klueska]

/milestone v1.30
/label lead-opted-in

[k8s-ci-robot]

@klueska: You must be a member of the kubernetes/milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your Milestone Maintainers Team and have them propose you as an additional delegate for this responsibility.

In response to this:

/milestone v1.30
/label lead-opted-in

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[johnbelamaric]

@SergeyKanzhelev can we get this added to the 1.30 milestone?

[johnbelamaric]

/label lead-opted-in

[k8s-ci-robot]

@klueska: Can not set label lead-opted-in: Must be member in one of these teams: [release-team-enhancements release-team-leads sig-api-machinery-leads sig-apps-leads sig-architecture-leads sig-auth-leads sig-autoscaling-leads sig-cli-leads sig-cloud-provider-leads sig-cluster-lifecycle-leads sig-contributor-experience-leads sig-docs-leads sig-instrumentation-leads sig-k8s-infra-leads sig-multicluster-leads sig-network-leads sig-node-leads sig-release-leads sig-scalability-leads sig-scheduling-leads sig-security-leads sig-storage-leads sig-testing-leads sig-windows-leads]

In response to this:

/milestone v1.30
/label lead-opted-in

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[SergeyKanzhelev]

/label lead-opted-in
/milestone v1.30

[salehsedghpour]

Hello @pohly 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 9th February 2024.

This enhancement is targeting for stage alpha for v1.30 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.30.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

For this KEP, we would just want to update the following but it's not required for alpha stage and it will serve for future stages:

• Make sure that the KEP 4381: add structured parameters for dynamic resource allocation #4384 is merged to the k/enhancements repo.

The status of this enhancement is marked as At risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well, I couldn't find any related PR regarding this KEP in k/enhancements. Thank you!

[salehsedghpour]

With all the requirements fulfilled this enhancement is now marked as tracked for the upcoming enhancements freeze 🚀

[drewhagen]

Hello @pohly 👋, 1.30 Docs Lead here.

Does this enhancement work planned for 1.30 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.30 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday February 22nd 2024 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

[hacktivist123]

Hello @pohly 👋 1.32 Docs Shadow here.

Does this enhancement work planned for 1.32 require any new docs or modifications to existing docs?
If so, please follow the steps here to open a PR against the dev-1.32 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday, October 24th 2024 18:00 PDT.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

[hacktivist123]

Hello, @pohly 👋 1.32 Docs Shadow here.

This is just a reminder to open a placeholder PR against dev-1.32 branch in the k/website repo for this (steps available here) for this KEP if it requires new or modifications to existing docs:

The deadline for this is Thursday, Oct 24 at 18:00 PDT.
Thanks! 🚀

[pohly]

@hacktivist123: I already listed such a PR under "docs" in the issue description.

[hacktivist123]

Oh didn't see that, sorry!

Thanks @pohly

[edithturn]

Hello @pohly ! 🙂 👋

This is a reminder to author your feature blog post!

To opt-in, let us know and open a Feature Blog placeholder PR against the website repository by 30th Oct 2024. For more information about writing a blog see the blog contribution guidelines.

Please feel free to reach out if you have any questions!

Thank you very much!
Edith from the v1.32 Shadow Communications Team

[tjons]

Hey again @pohly 👋 v1.32 Enhancements team here,

Just checking in as we approach code freeze at 02:00 UTC Friday 8th November 2024 / 19:00 PDT Thursday 7th November 2024 .

Here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze (and we need to update the Issue description to include all the related PRs of this KEP):

• DRA 1.32 API: promotion to beta kubernetes#127511

Additionally, please let me know if there are any other PRs in k/k not listed in the description or not linked with this GitHub issue that we should track for this KEP, so that we can maintain accurate status.

The status of this enhancement is marked as at risk for code freeze.

If you anticipate missing code freeze, you can file an exception request in advance. Thank you!

[tjons]

Hello @pohly 👋, Enhancements team here.

With all the implementation(code related) PRs merged as per the issue description:

• DRA resourceclaims: maintain metric of total and allocated claims kubernetes#127661
• DRA API: AdminAccess in DeviceRequestAllocationResult + DRAAdminAccess feature gate kubernetes#127266
• DRA API: implement CEL cost limit kubernetes#128101
• Kubelet: add DRA latency metrics kubernetes#127146
• DRA resourceclaims: maintain metric of total and allocated claims kubernetes#127661
• DRA 1.32 API: promotion to beta kubernetes#127511

This enhancement is now marked as tracked for code freeze for the 1.32 Code Freeze!

Please note that KEPs targeting stable need to have the status field marked as implemented in the kep.yaml file after code PRs are merged and the feature gates are removed.

[wzshiming]

The v1beta1.Device is the same name as the existing one. Maybe we should change the name?

kubernetes/kubernetes#128730 (comment)

[pohly]

/wg device-management

[haircommander]

/reopen

A github workflow autoclosed when I set to done 🙃 sorry for the noise

[k8s-ci-robot]

@haircommander: Reopened this issue.

In response to this:

/reopen

A github workflow autoclosed when I set to done 🙃 sorry for the noise

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[dipesh-rawat]

Hello @pohly 👋, 1.33 Enhancements Lead here.

I’m closing milestone 1.32 now. If you'd like to work on this enhancement in v1.33, please have the SIG lead opt-in by adding the lead-opted-in label, which ensures it gets added to the tracking board. Also, please set the milestone to v1.33 using /milestone v1.33.
Thanks!

/remove-label lead-opted-in
/milestone clear