[Feature] Kubernetes native scheduler support for single-controller mode

[d3c3mber]

Checklist

• This feature will maintain backward compatibility with the current APIs in
  areal/api/. If not, please raise a refactor issue first.

Background

AReaL single-controller mode supports the local scheduler only. This feature proposal plans to add K8s-native scheduler support for AReaL, for running RL jobs (for both LLM RL training and Agentic RL training) in large-scale clusters in a cloud-native way.

Potential Solution

We plan to add a k8s based scheduler/controller for scheduling and managing AReaL master component and worker components.
It will include the following features:

• cloud native way for running areal components
• support for users to adopt customized code from git for rl framework and rollout/train framework

Additional Information

(Add any relevant context, references, or supporting data here.)

[garrett4wade]

Hi @d3c3mber, thanks for putting together this proposal! Sorry for the delayed response. Adding K8s-native scheduler support for AReaL will be a valuable enhancement for large-scale deployments.

I wanted to provide some context that might help clarify the scope and design approach:

1. Component Mapping: The scheduling_spec in TrainEngineConfig supports a two-element tuple structure. The first spec defines the "worker" pod (CPU-only), and the second defines the "engine" pod (GPU container). While this distinction doesn't matter in local environments, it's designed to enable proper K8s resource allocation.

2. Controller Deployment Model: We envision the controller/scheduler operating on a local node that submits jobs to the cluster, rather than running as a K8s job itself. This keeps the control plane outside the cluster for simpler management.

3. Integration Flexibility: There are minimal restrictions on the scheduler's init API—the main requirement is that it works correctly. By "works correctly," I mean we should be able to swap out the scheduler in [this example script](https://github.com/inclusionAI/AReaL/blob/main/examples/single-controller/gsm8k_grpo.py) with a properly initialized K8s scheduler, and training should run without issues.

Let me know if you need any clarification on these points. Looking forward to seeing the next steps!

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity within the last 14 days.

If this issue is still relevant, please add a comment to keep it open. Otherwise, it will be automatically closed in 16 days.

Thank you for your contribution!