[Core] Resource Isolation Milestone 1 Implementation Tracker

[israbbani]

Implement support for isolating ray system processes from application processes using cgroupv2.

Work Items

• Create an API for passing cgroup configuration into ray ([core] Adding user facing API for resource isolation #51865).
• Implement CI support for running cgroup tests ([ci] Enable Cgroup support in CI for core #51454).
• Implement a sysfs driver for cgroup operations with tests. ([core] (cgroups 1/n) Adding a sys/fs filesystem driver to perform cgroup operations. #54898).
• Implement integration tests for the sysfs driver. ([core] (cgroups 2/n) adding integration tests for the cgroup sysfs driver.  #55063).
• Implement a cgroup manager that uses the cgroup driver to check invariants, create subcgroups, move processes, enable controllers and resource limits.
  • [core] (cgroups 3/n) Creating CgroupManager to setup Ray's cgroup hierarchy and clean it up #56186
  • [core] (cgroups 4/n) adding constraint bounds checking to the CgroupManager. #56246
• Implement cgroup cleanup in cgroup manager.
  • [core] (cgroups 5/n) Adding clean up methods to CgroupDriverInterface and SysFsCgroupDriver #56255
  • [core] (cgroups 6/n) CgroupManager cleans up the entire cgroup hierarchy in reverse order when destroyed. #56260
• Implement process migration for system processes and worker processes into the correct cgroup before or on startup.
  • [core] (cgroups 7/n) cleaning up old cgroup integration code for raylet and core worker #56285
  • [core] (cgroups 8/n) Wiring CgroupManager into the raylet. #56297
  • [core] (cgroups 9/n) end-to-end integration of cgroups with ray start. #56352
  • [core] (cgroups 10/n) Adding support in CgroupManager and CgroupDriver to move processes into system cgroup #56446
  • [core] (cgroups 11/n) Raylet will move system processes into cgroup on startup #56522
  • [core] (cgroups 12/n) Raylet will start worker processes in the application cgroup #56549
• Cleanup old cgroup code and associated TODOs.
  • [core] (cgroups 13/n) Deleting old cgroups implementation #56909
• Add a ProcessIsolationFactory as described here. Clean up all public bazel targets.
  • [core] (cgroups 14/n) Clean up bazel targets and support cross-platform build. #57244
• Add a user cgroup for all non-ray processes.
  • [core] (cgroups 15/n) Adding a user cgroup subtree for non-ray processes. #57269
• Tune defaults for --system-reserved-cpu and --system-reserved-memory.
  • [core] (cgroups 16/n) Changing the algorithm for default values for system reserved resources #57653
• Moving the driver and dashboard subprocesses into the system cgroup
  • [core] (cgroups 18/n) Moving the driver into the workers cgroup. #57776
  • [core] (cgroups 20/n) Move dashboard api server's subprocesses into system cgroup #57864
• Bug fixes
  • [core] (cgroups 17/n) Removing controller check when adding a constraint #57731
  • [core] (cgroups 19/n) Allow fractions when getting the number of CPUs to calculate weights #57800
• Add user-facing documentation for enabling resource isolation on VMs and containers.
• Update Log messages to cross-link to user-facing documentation.
• Attempt to move all cgroup related functionality into its own namespace to see how it plays with developer ergonomics.