[core] (cgroups 10/n) Adding support in CgroupManager and CgroupDriver to move processes into system cgroup #56446
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
to perform cgroup operations. Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: irabbani <irabbani@anyscale.com>
instead of clone for older kernel headers < 5.7 (which is what we have in CI) Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: irabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Ibrahim Rabbani <israbbani@gmail.com>
…irabbani/cgroups-1
Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: irabbani <irabbani@anyscale.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Ibrahim Rabbani <israbbani@gmail.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Ibrahim Rabbani <israbbani@gmail.com>
…irabbani/cgroups-1
Signed-off-by: irabbani <irabbani@anyscale.com>
fix CI. Signed-off-by: irabbani <irabbani@anyscale.com>
Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
…irabbani/cgroups-9
Signed-off-by: irabbani <israbbani@gmail.com>
Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com>
ZacAttack
reviewed
Sep 16, 2025
| /** | ||
| Moves the process into the system leaf cgroup (@see kLeafCgroupName). | ||
| To move the pid, the process must have read, write, and execute permissions for the |
Contributor
There was a problem hiding this comment.
nit: Add a colon at the end of the line since it preceeds a list
ZacAttack
approved these changes
Sep 17, 2025
ZacAttack
approved these changes
Sep 17, 2025
edoakes
approved these changes
Sep 17, 2025
edoakes
pushed a commit
that referenced
this pull request
Sep 18, 2025
…6626) Broken in #56446. This should stop being possible once there's a single cgroups target exported (as highlighted in #54703). I've fixed the broken build and I've added a temporary test target that builds the noop implementations as part of Linux CI so it gets caught in premerge. --------- Signed-off-by: irabbani <israbbani@gmail.com>
zma2
pushed a commit
to zma2/ray
that referenced
this pull request
Sep 23, 2025
…r to move processes into system cgroup (ray-project#56446) This PR stacks on ray-project#56352 . For more details about the resource isolation project see ray-project#54703. This PR the following functions to move a process into the system cgroup: * CgroupManagerInterface::AddProcessToSystemCgroup * CgroupDriverInterface::AddProcessToCgroup I've also added integration tests for SysFsCgroupDriver and unit tests for CgroupManager. Let me explain how these APIs will be used. In the next PR, the raylet will * be passed a list of pids of system processes that are started before the raylet starts and need to be moved into the system cgroup (e.g. gcs_server) * call CgroupManagerInterface::AddProcessToSystemCgroup for each of these pids to move them into the system cgroup. --------- Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Zhiqiang Ma <zhiqiang.ma@intel.com>
zma2
pushed a commit
to zma2/ray
that referenced
this pull request
Sep 23, 2025
…y-project#56626) Broken in ray-project#56446. This should stop being possible once there's a single cgroups target exported (as highlighted in ray-project#54703). I've fixed the broken build and I've added a temporary test target that builds the noop implementations as part of Linux CI so it gets caught in premerge. --------- Signed-off-by: irabbani <israbbani@gmail.com> Signed-off-by: Zhiqiang Ma <zhiqiang.ma@intel.com>
ZacAttack
pushed a commit
to ZacAttack/ray
that referenced
this pull request
Sep 24, 2025
…r to move processes into system cgroup (ray-project#56446) This PR stacks on ray-project#56352 . For more details about the resource isolation project see ray-project#54703. This PR the following functions to move a process into the system cgroup: * CgroupManagerInterface::AddProcessToSystemCgroup * CgroupDriverInterface::AddProcessToCgroup I've also added integration tests for SysFsCgroupDriver and unit tests for CgroupManager. Let me explain how these APIs will be used. In the next PR, the raylet will * be passed a list of pids of system processes that are started before the raylet starts and need to be moved into the system cgroup (e.g. gcs_server) * call CgroupManagerInterface::AddProcessToSystemCgroup for each of these pids to move them into the system cgroup. --------- Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: zac <zac@anyscale.com>
ZacAttack
pushed a commit
to ZacAttack/ray
that referenced
this pull request
Sep 24, 2025
…y-project#56626) Broken in ray-project#56446. This should stop being possible once there's a single cgroups target exported (as highlighted in ray-project#54703). I've fixed the broken build and I've added a temporary test target that builds the noop implementations as part of Linux CI so it gets caught in premerge. --------- Signed-off-by: irabbani <israbbani@gmail.com> Signed-off-by: zac <zac@anyscale.com>
elliot-barn
pushed a commit
that referenced
this pull request
Sep 24, 2025
…r to move processes into system cgroup (#56446) This PR stacks on #56352 . For more details about the resource isolation project see #54703. This PR the following functions to move a process into the system cgroup: * CgroupManagerInterface::AddProcessToSystemCgroup * CgroupDriverInterface::AddProcessToCgroup I've also added integration tests for SysFsCgroupDriver and unit tests for CgroupManager. Let me explain how these APIs will be used. In the next PR, the raylet will * be passed a list of pids of system processes that are started before the raylet starts and need to be moved into the system cgroup (e.g. gcs_server) * call CgroupManagerInterface::AddProcessToSystemCgroup for each of these pids to move them into the system cgroup. --------- Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
elliot-barn
pushed a commit
that referenced
this pull request
Sep 24, 2025
…6626) Broken in #56446. This should stop being possible once there's a single cgroups target exported (as highlighted in #54703). I've fixed the broken build and I've added a temporary test target that builds the noop implementations as part of Linux CI so it gets caught in premerge. --------- Signed-off-by: irabbani <israbbani@gmail.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
marcostephan
pushed a commit
to marcostephan/ray
that referenced
this pull request
Sep 24, 2025
…r to move processes into system cgroup (ray-project#56446) This PR stacks on ray-project#56352 . For more details about the resource isolation project see ray-project#54703. This PR the following functions to move a process into the system cgroup: * CgroupManagerInterface::AddProcessToSystemCgroup * CgroupDriverInterface::AddProcessToCgroup I've also added integration tests for SysFsCgroupDriver and unit tests for CgroupManager. Let me explain how these APIs will be used. In the next PR, the raylet will * be passed a list of pids of system processes that are started before the raylet starts and need to be moved into the system cgroup (e.g. gcs_server) * call CgroupManagerInterface::AddProcessToSystemCgroup for each of these pids to move them into the system cgroup. --------- Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Marco Stephan <marco@magic.dev>
marcostephan
pushed a commit
to marcostephan/ray
that referenced
this pull request
Sep 24, 2025
…y-project#56626) Broken in ray-project#56446. This should stop being possible once there's a single cgroups target exported (as highlighted in ray-project#54703). I've fixed the broken build and I've added a temporary test target that builds the noop implementations as part of Linux CI so it gets caught in premerge. --------- Signed-off-by: irabbani <israbbani@gmail.com> Signed-off-by: Marco Stephan <marco@magic.dev>
elliot-barn
pushed a commit
that referenced
this pull request
Sep 27, 2025
…r to move processes into system cgroup (#56446) This PR stacks on #56352 . For more details about the resource isolation project see #54703. This PR the following functions to move a process into the system cgroup: * CgroupManagerInterface::AddProcessToSystemCgroup * CgroupDriverInterface::AddProcessToCgroup I've also added integration tests for SysFsCgroupDriver and unit tests for CgroupManager. Let me explain how these APIs will be used. In the next PR, the raylet will * be passed a list of pids of system processes that are started before the raylet starts and need to be moved into the system cgroup (e.g. gcs_server) * call CgroupManagerInterface::AddProcessToSystemCgroup for each of these pids to move them into the system cgroup. --------- Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
elliot-barn
pushed a commit
that referenced
this pull request
Sep 27, 2025
…6626) Broken in #56446. This should stop being possible once there's a single cgroups target exported (as highlighted in #54703). I've fixed the broken build and I've added a temporary test target that builds the noop implementations as part of Linux CI so it gets caught in premerge. --------- Signed-off-by: irabbani <israbbani@gmail.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
dstrodtman
pushed a commit
that referenced
this pull request
Oct 6, 2025
…r to move processes into system cgroup (#56446) This PR stacks on #56352 . For more details about the resource isolation project see #54703. This PR the following functions to move a process into the system cgroup: * CgroupManagerInterface::AddProcessToSystemCgroup * CgroupDriverInterface::AddProcessToCgroup I've also added integration tests for SysFsCgroupDriver and unit tests for CgroupManager. Let me explain how these APIs will be used. In the next PR, the raylet will * be passed a list of pids of system processes that are started before the raylet starts and need to be moved into the system cgroup (e.g. gcs_server) * call CgroupManagerInterface::AddProcessToSystemCgroup for each of these pids to move them into the system cgroup. --------- Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com> Signed-off-by: Douglas Strodtman <douglas@anyscale.com>
dstrodtman
pushed a commit
to dstrodtman/ray
that referenced
this pull request
Oct 6, 2025
…y-project#56626) Broken in ray-project#56446. This should stop being possible once there's a single cgroups target exported (as highlighted in ray-project#54703). I've fixed the broken build and I've added a temporary test target that builds the noop implementations as part of Linux CI so it gets caught in premerge. --------- Signed-off-by: irabbani <israbbani@gmail.com> Signed-off-by: Douglas Strodtman <douglas@anyscale.com>
justinyeh1995
pushed a commit
to justinyeh1995/ray
that referenced
this pull request
Oct 20, 2025
…r to move processes into system cgroup (ray-project#56446) This PR stacks on ray-project#56352 . For more details about the resource isolation project see ray-project#54703. This PR the following functions to move a process into the system cgroup: * CgroupManagerInterface::AddProcessToSystemCgroup * CgroupDriverInterface::AddProcessToCgroup I've also added integration tests for SysFsCgroupDriver and unit tests for CgroupManager. Let me explain how these APIs will be used. In the next PR, the raylet will * be passed a list of pids of system processes that are started before the raylet starts and need to be moved into the system cgroup (e.g. gcs_server) * call CgroupManagerInterface::AddProcessToSystemCgroup for each of these pids to move them into the system cgroup. --------- Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
justinyeh1995
pushed a commit
to justinyeh1995/ray
that referenced
this pull request
Oct 20, 2025
…y-project#56626) Broken in ray-project#56446. This should stop being possible once there's a single cgroups target exported (as highlighted in ray-project#54703). I've fixed the broken build and I've added a temporary test target that builds the noop implementations as part of Linux CI so it gets caught in premerge. --------- Signed-off-by: irabbani <israbbani@gmail.com>
landscapepainter
pushed a commit
to landscapepainter/ray
that referenced
this pull request
Nov 17, 2025
…r to move processes into system cgroup (ray-project#56446) This PR stacks on ray-project#56352 . For more details about the resource isolation project see ray-project#54703. This PR the following functions to move a process into the system cgroup: * CgroupManagerInterface::AddProcessToSystemCgroup * CgroupDriverInterface::AddProcessToCgroup I've also added integration tests for SysFsCgroupDriver and unit tests for CgroupManager. Let me explain how these APIs will be used. In the next PR, the raylet will * be passed a list of pids of system processes that are started before the raylet starts and need to be moved into the system cgroup (e.g. gcs_server) * call CgroupManagerInterface::AddProcessToSystemCgroup for each of these pids to move them into the system cgroup. --------- Signed-off-by: Ibrahim Rabbani <irabbani@anyscale.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
landscapepainter
pushed a commit
to landscapepainter/ray
that referenced
this pull request
Nov 17, 2025
…y-project#56626) Broken in ray-project#56446. This should stop being possible once there's a single cgroups target exported (as highlighted in ray-project#54703). I've fixed the broken build and I've added a temporary test target that builds the noop implementations as part of Linux CI so it gets caught in premerge. --------- Signed-off-by: irabbani <israbbani@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR stacks on #56352 .
For more details about the resource isolation project see #54703.
This PR the following functions to move a process into the system cgroup:
I've also added integration tests for SysFsCgroupDriver and unit tests for CgroupManager.
Let me explain how these APIs will be used. In the next PR, the raylet will