KEP-5547: Add workloadRef in the Job API #5548

helayoty · 2025-09-19T15:46:45Z

One-line PR description: Support associating a Job with a Workload via explicit field
Issue link: WAS: Integrate Workload API with true controllers #5547

/sig scheduling
/sig apps
/stage alpha

Other comments:

kannon92 · 2025-09-19T16:24:10Z

To save paperwork, why would we not include this as part of the gang scheduling KEP?

These KEPs would need to go in tandem and if one goes without the other we would have to revert.

helayoty · 2025-09-19T16:51:43Z

To save paperwork, why would we not include this as part of the gang scheduling KEP?

These KEPs would need to go in tandem and if one goes without the other we would have to revert.

After discussion with SIG Apps, they explicitly requested the changes to the Job API (or any other API) be handled through a dedicated KEP. This aligns with how API changes are typically tracked, even if the field is optional and tied to a feature gate shared with the broader gang scheduling effort.

To keep the implementations in sync and avoid divergence, we’re planning to use the same feature gate as the gang scheduling KEP. This way, the feature behaves atomically at runtime (one can’t go live without the other), even though it’s tracked in two KEPs for process and ownership clarity.

I've added link to both KEPs to each other and clearly document the dependency if that helps reviewers track the connection.

k8s-ci-robot · 2025-09-30T12:12:27Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: helayoty
Once this PR has been reviewed and has the lgtm label, please assign soltysh for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

keps/prod-readiness/OWNERS
keps/sig-apps/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Signed-off-by: Heba Elayoty <heelayot@microsoft.com>

soltysh

I've left several comments, but the biggest one that's currently blocking the further progress on this document is the one about workloadRef vs the workload field in pod.spec. I believe before going further with this proposal we should identify the direction in the light of future evolution of [KEP-4671](https://github.com/kubernetes/enhancements/pull/5558]

soltysh · 2025-10-01T12:04:34Z

keps/prod-readiness/sig-apps/5547.yaml

@@ -0,0 +1,3 @@
+kep-number: 5547
+alpha:
+  approver: "@soltysh"


Based on PRR dashboard this will be done by @johnbelamaric . Ideally we should avoid having SIG approver and PRR approver be the same person. So please change accordingly.

soltysh · 2025-10-01T12:06:09Z

keps/sig-apps/5547-workload-ref-in-job/README.md

+- [ ] (R) Production readiness review approved
+- [ ] "Implementation History" section is up-to-date for milestone
+- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
+- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes


Check the following boxes:

Design details are appropriately documented

Test plan is in place...

Production readiness review completed

Implementation History...

soltysh · 2025-10-01T12:09:59Z

keps/sig-apps/5547-workload-ref-in-job/README.md

+
+### Goals
+
+- Introduce a new optional `workloadRef` field in the `JobSpec`, allowing a Job to declare an explicit association with a higher-level workload object.


I've asked similar question in the gang scheduling KEP, where there's a proposal to expand pod.spec with workload information. This means Job's pod template will have the ability to set this field as well, why do we want to introduce workloadRef in that case?

Basically with that new workload field in pod.spec an example Job could look like:

spec: template: spec: + workload: + name: job-1 ...

In that other doc, there is a statement discussing potential future evolution where a controller would infer intentions from a workload and create the Workload resource accordingly. Although I've expressed some concerns around that idea.

Linking to my reply on the main KEP so we have one place for the whole conversation.
#5558 (comment)

soltysh · 2025-10-01T12:14:29Z

keps/sig-apps/5547-workload-ref-in-job/README.md

+  - Will enabling / disabling the feature require downtime of the control
+    plane?
+  - Will enabling / disabling the feature require downtime or reprovisioning
+    of a node?


Hmmm... I don't think re-using the same feature gate as KEP-4671 is advisable. I'd expect separate one just for that field. You can express that this FG relies on the other ones, we've done that in the past (see most of DRA).

soltysh · 2025-10-01T12:15:56Z

keps/sig-apps/5547-workload-ref-in-job/kep.yaml

+owning-sig: sig-apps
+participating-sigs:
+  - sig-scheduling
+  - sig-apps


Nit: no need to mention sig-apps again.

soltysh · 2025-10-01T12:16:35Z

keps/sig-apps/5547-workload-ref-in-job/kep.yaml

+disable-supported: true
+
+metrics:
+  - TDB


job_sync_duration_seconds as mentioned in SLI section?

soltysh · 2025-10-01T12:16:51Z

keps/sig-apps/5547-workload-ref-in-job/kep.yaml

+    components:
+      - kube-apiserver
+      - kube-scheduler
+      - kube-controller-manager


This section must match what you have in the main document.

janetkuo · 2025-10-01T20:33:30Z

keps/sig-apps/5547-workload-ref-in-job/kep.yaml

+milestone:
+  alpha: "v1.35"
+  beta: TDB
+  stable: TDB


janetkuo · 2025-10-01T20:35:49Z

keps/sig-apps/5547-workload-ref-in-job/README.md

+
+## Summary
+
+Introduce a new optional field in the Job API spec to explicitly associate a Job with a Workload object, enabling safe coordination between workload-aware (Gang) scheduling and job controllers without introducing race conditions or forcing the scheduler to perform controller duties.


Given that Workload is widely used in SIG Apps, i.e. core workloads API, suggest briefly explaining and clarifying what you mean by Workload here.

janetkuo · 2025-10-01T20:38:06Z

keps/sig-apps/5547-workload-ref-in-job/README.md

+
+NA
+
+[^1]: The Kubernetes community uses the term "gang scheduling" to mean "all-or-nothing scheduling of a set of pods" [1,2,3,4,5,6,7,8,9,10,11,12,13]. In the Kubernetes context, it does not imply time-multiplexing (in contrast to prior academic work such as [Feitelson and Rudolph](https://doi.org/10.1016/0743-7315(92)90014-E), and in contrast to [Slurm Gang Scheduling](https://slurm.schedmd.com/gang_scheduling.html)).  


What does "1,2,3,4,5,6,7,8,9,10,11,12,13" mean?

janetkuo · 2025-10-01T21:00:54Z

/hold

Holding to ensure that this is merged after #5558 is merged

soltysh · 2025-10-08T09:47:16Z

@helayoty after all the discussions that happened in https://github.com/kubernetes/enhancements/pull/5558/files#r2410167467 I believe we can remove this KEP from 1.35. Eventual future changes to Job API will happen at a later stage, but we're not sure what those fields are, yet. So either we can close #5547 entirely, or you'll rename it to something more appropriate like "WAS: extend Job API to support workload" and we'll pick it up once we know what we actually want to achieve.

soltysh · 2025-10-13T09:22:02Z

Based on my previous comment, and discussions with @helayoty we're not going to pursuit this option for 1.35. So I'm going to close this PR and will opt-out the feature from 1.35 entirely. We can revisit it at a later point in time, and based on the work happening in #4671 we'll reconsider our options.

/close

k8s-ci-robot · 2025-10-13T09:22:08Z

@soltysh: Closed this PR.

Details

In response to this:

Based on my previous comment, and discussions with @helayoty we're not going to pursuit this option for 1.35. So I'm going to close this PR and will opt-out the feature from 1.35 entirely. We can revisit it at a later point in time, and based on the work happening in #4671 we'll reconsider our options.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

github-project-automation bot added this to SIG Scheduling and SIG Apps Sep 19, 2025

github-project-automation bot moved this to Needs Triage in SIG Apps Sep 19, 2025

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory labels Sep 19, 2025

k8s-ci-robot requested review from kow3ns and soltysh September 19, 2025 15:46

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 19, 2025

helayoty mentioned this pull request Sep 19, 2025

WAS: Integrate Workload API with true controllers #5547

Open

4 tasks

helayoty moved this from Needs Triage to In Progress in SIG Apps Sep 19, 2025

helayoty moved this to In Progress in SIG Scheduling Sep 19, 2025

helayoty force-pushed the helayoty/workload-job-api branch from 61fdfb2 to 6e6e5fb Compare September 19, 2025 20:21

helayoty force-pushed the helayoty/workload-job-api branch 2 times, most recently from c33947a to a22d09c Compare September 30, 2025 12:12

helayoty force-pushed the helayoty/workload-job-api branch 4 times, most recently from 635181f to fe7e012 Compare September 30, 2025 12:32

Add workload ref to job spec kep

b9f9156

Signed-off-by: Heba Elayoty <heelayot@microsoft.com>

helayoty force-pushed the helayoty/workload-job-api branch from fe7e012 to b9f9156 Compare September 30, 2025 14:47

helayoty changed the title ~~[WIP] KEP-5547: Add workloadRef in the Job API~~ KEP-5547: Add workloadRef in the Job API Sep 30, 2025

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 30, 2025

soltysh reviewed Oct 1, 2025

View reviewed changes

janetkuo reviewed Oct 1, 2025

View reviewed changes

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 1, 2025

helayoty mentioned this pull request Oct 1, 2025

KEP-4671 : Gang Scheduling #5558

Merged

k8s-ci-robot closed this Oct 13, 2025

github-project-automation bot moved this from In Progress to Closed in SIG Apps Oct 13, 2025

andreyvelich mentioned this pull request Oct 15, 2025

KEP for GangConfig (to support Gang Scheduling) kubernetes-sigs/jobset#1068

Open

helayoty moved this from In Progress to Closed in SIG Scheduling Oct 16, 2025


		### Goals

		- Introduce a new optional `workloadRef` field in the `JobSpec`, allowing a Job to declare an explicit association with a higher-level workload object.


		## Summary

		Introduce a new optional field in the Job API spec to explicitly associate a Job with a Workload object, enabling safe coordination between workload-aware (Gang) scheduling and job controllers without introducing race conditions or forcing the scheduler to perform controller duties.


		NA

		[^1]: The Kubernetes community uses the term "gang scheduling" to mean "all-or-nothing scheduling of a set of pods" [1,2,3,4,5,6,7,8,9,10,11,12,13]. In the Kubernetes context, it does not imply time-multiplexing (in contrast to prior academic work such as [Feitelson and Rudolph](https://doi.org/10.1016/0743-7315(92)90014-E), and in contrast to [Slurm Gang Scheduling](https://slurm.schedmd.com/gang_scheduling.html)).

KEP-5547: Add workloadRef in the Job API #5548

KEP-5547: Add workloadRef in the Job API #5548

Uh oh!

Conversation

helayoty commented Sep 19, 2025

Uh oh!

kannon92 commented Sep 19, 2025

Uh oh!

helayoty commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Sep 30, 2025

Uh oh!

soltysh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

helayoty Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

janetkuo commented Oct 1, 2025

Uh oh!

soltysh commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

soltysh commented Oct 13, 2025

Uh oh!

k8s-ci-robot commented Oct 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

helayoty commented Sep 19, 2025 •

edited

Loading

helayoty Oct 1, 2025 •

edited

Loading

soltysh commented Oct 8, 2025 •

edited

Loading