Skip to content

Sandbox execution on Linux - #2241

Merged
chrisstaite-menlo merged 1 commit into
TraceMachina:mainfrom
chrisstaite:container_run
Mar 26, 2026
Merged

chrisstaite-menlo merged 1 commit into
TraceMachina:mainfrom
chrisstaite:container_run

Conversation

@chrisstaite

@chrisstaite chrisstaite commented Mar 23, 2026 •

Copy link
Copy Markdown
Contributor

Description

When executing on Linux there are issues with running as the init process and not reaping children. This is incompatible with the Tokio runtime.

Therefore, in the pre_exec we fork a new process which becomes the init process for that execution. We can do this by making use of user namespacing. Since we're namespacing anyway, include the UTS and IPC namespaces to isolate the execution even more and make it more reproducible.

Type of change

Please delete options that aren't relevant.

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How Has This Been Tested?

Tests are passing with and without the activation of the feature.

Checklist

  • Updated documentation if needed
  • Tests added/amended
  • bazel test //... passes locally
  • PR is contained in a single commit, using git amend see some docs

This change is Reviewable

Comment thread nativelink-worker/src/local_worker.rs Outdated
@chrisstaite

Copy link
Copy Markdown
Contributor Author

Don't know why the mongo store tests are failing, they work locally.

@chrisstaite

Copy link
Copy Markdown
Contributor Author

They were apparently depending on the fact that sandboxing was broken in Bazel due to the apparmor_restrict_unprivileged_userns setting. I have re-broken this for the Bazel runs, but it's still enabled for the Cargo version to ensure it's an actually tested code path.

@chrisstaite

Copy link
Copy Markdown
Contributor Author

I've refactored into a separate module and written full safety comments for each unsafe call. I'm happy with this now, it will absolutely resolve the issue of #225 if it's activated.

Comment thread nativelink-config/src/cas_server.rs
@palfrey

palfrey commented Mar 25, 2026

Copy link
Copy Markdown
Member

/build-image nativelink-worker-init

@palfrey

palfrey commented Mar 25, 2026

Copy link
Copy Markdown
Member

/build-image

@github-actions

Copy link
Copy Markdown
Contributor

Image built and pushed!

ghcr.io/TraceMachina/nativelink-worker-init:0b80b15

@github-actions

Copy link
Copy Markdown
Contributor

Image built and pushed!

ghcr.io/TraceMachina/nativelink:0b80b15

@chrisstaite

Copy link
Copy Markdown
Contributor Author

I've updated this to send SIGTERM when namespaced to avoid the zombie issue.

When executing on Linux there are issues with running as the init
process and not reaping children.  This is incompatible with the
Tokio runtime.

Therefore, in the pre_exec we fork a new process which becomes the
init process for that execution.  We can do this by making use of
user namespacing.  Since we're namespacing anyway, include the UTS
and IPC namespaces to isolate the execution even more and make it
more reproducible.
@chrisstaite

Copy link
Copy Markdown
Contributor Author

I've created zombie tests and also made sure the kill signal is raised correctly from the child. So that's nice. Also, the rebase fix-up is complete too. All ready to merge now.

This branch was previously deployed

1 inactive deployment
production — e202c0ff Deployed Mar 26, 2026 by chrisstaite via Web Platform Deployment / macos-15 #2283
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants