feat(manager): configure executor worker concurrency, harden post-install by kp2pml30 · Pull Request #43 · genlayerlabs/genvm-manager

kp2pml30 · 2026-09-17T09:49:10Z

Problem and outcome

Two independent threads, both manager-side with executor projections.

Worker control. The manager could not tell an executor how many workers to
use, so a nondeterministic block always ran fully concurrent. The config now
carries worker concurrency, it is passed in the execution input, and the
executor reads it at startup — a single-worker run becomes deterministic in
print ordering. v0.2.x warns on the restriction it cannot honor.

Post-install hardening. The installer built its venv with pip install
against version pins only, so the wheels it fetched were unverified. It also
carried an executor-download fallback that has never been able to run: the
executor repo publishes no releases, and no line pinned the sha256 the code
demanded.

Non-goals: verifying the v0.2.x legacy runner registry (its Nix-base32 hashes
are checked by that executor's own check, and the gap is now logged loudly);
acting on --arch beyond accepting it.

Implementation and validation

implementation/src/manager/run.rs + crates/modules-interfaces — worker
control in the execution input, covered by run_test.rs
install/lib/python/post-install/ — --require-hashes, every pin carrying
the sha256 of every distribution PyPI publishes, refreshed by
support/scripts/refresh-requirements-hashes.py. The --use-patchelf
variant drops the whole lief block, hashes included
Executor download removed end to end: download_executor,
--executor-download, executor_download_urls, the executor-sha256
manifest ingestion and its tests. --arch stays accepted — out-of-tree
installers forward it — but nothing reads it
Validation: 66/66 genvm-tool unit tests; manager Rust tests; a real venv
built through --require-hashes; manifest.build on this tree emits both
lines with available_after only

Rollback: the post-install commits are independent of the worker-control ones
and revert cleanly on their own.

Summary by CodeRabbit

New Features
- Added configurable control over overlapping deterministic execution and nondeterministic validation, enabled by default.
- Added an unsafe-run override for this scheduling behavior.
- Manager log collection now waits for complete log output, including early executor exits.
Install & Security
- Executor packages are provided through platform release assets rather than post-install downloads.
- Python dependency installation now requires SHA-256 hashes.
Breaking Changes
- Documented new v0.3 resource limits for memory usage, loaded runners, and virtual filesystem path lengths.

…s 🔒️

coderabbitai · 2026-09-17T09:49:24Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: bc8d7163-9b16-4994-8554-d1677bfd2e34

📥 Commits

Reviewing files that changed from the base of the PR and between be558e0 and c2caddc.

⛔ Files ignored due to path filters (9)

docs/schemas/default-config.json is excluded by !**/*.json
implementation/src/manager/run_test.rs is excluded by !**/*_test.rs
install/lib/python/post-install/requirements.txt is excluded by !**/*.txt
support/tools/genvm-tool/genvm_tool/manifest.py is excluded by !support/tools/genvm-tool/**
support/tools/genvm-tool/unit_tests/test_manifest.py is excluded by !support/tools/genvm-tool/**
support/tools/genvm-tool/unit_tests/test_post_install_executor_download.py is excluded by !support/tools/genvm-tool/**
tests/runner/genvm_tool_plugins/integration.py is excluded by !**/tests/**
tests/runner/origin/base_host.py is excluded by !**/tests/**
tests/system/cross-major/test.py is excluded by !**/tests/**

📒 Files selected for processing (23)

.agents/skills/agentic-fuzzing/SKILL.md
.genvm-monorepo-root
crates/modules-interfaces/src/domain.rs
docs/contributing/howto/committing/submodules.md
docs/contributing/howto/releasing/release-build.md
docs/contributing/howto/releasing/versioning.md
docs/contributing/howto/testing/integration.md
docs/website/src/impl-spec/appendix/log-record.rst
docs/website/src/impl-spec/appendix/manager-api.yaml
docs/website/src/impl-spec/appendix/manager-socket.rst
docs/website/src/spec/changelog.rst
executors/v0.2.x
executors/v0.3.x
implementation/src/manager/mod.rs
implementation/src/manager/run.rs
install/bin/genvm-post-install
install/config/genvm-manager.yaml
install/lib/python/post-install/__main__.py
install/lib/python/post-install/create_venv.py
support/ci/pipelines/checks.py
support/ci/unit_tests/test_cargo_clippy.py
support/manifest-base.yaml
support/scripts/refresh-requirements-hashes.py

💤 Files with no reviewable changes (1)

support/manifest-base.yaml

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

Walkthrough

The manager adds configurable worker concurrency and coordinated executor log collection. The installer removes executor downloads and enforces hashed Python requirements. Release metadata, documentation, executor references, and CI summaries are updated.

Changes

Manager execution

Layer / File(s)	Summary
Worker concurrency configuration `crates/modules-interfaces/src/domain.rs`, `implementation/src/manager/mod.rs`, `implementation/src/manager/run.rs`, `install/config/genvm-manager.yaml`, `docs/website/src/impl-spec/appendix/*`	`allow_two_workers` is added to configuration, execution data, request overrides, and API documentation. The default is `true`.
Concurrent executor process and log handling `implementation/src/manager/run.rs`, `docs/website/src/impl-spec/appendix/log-record.rst`	Executor log reading now runs with process handling and completes through EOF before the process result returns. Log-pipe setup uses `std::io::pipe()` and an executor-only inheritable write end.

Installer integrity

Layer / File(s)	Summary
Bundled executor installation `install/lib/python/post-install/__main__.py`, `support/manifest-base.yaml`	Post-install no longer downloads missing executor archives. The executor download option and manifest URL template are removed.
Hashed Python requirements `install/lib/python/post-install/create_venv.py`, `support/scripts/refresh-requirements-hashes.py`	Virtual-environment installation now uses `--require-hashes`. A script refreshes requirement hashes from PyPI and supports check-only mode.
Exact requirement removal `install/bin/genvm-post-install`	The patchelf requirement generation removes only the exact pinned `lief` requirement and its continuation lines.

Release alignment

Layer / File(s)	Summary
Release source and version updates `.genvm-monorepo-root`, `executors/v0.2.x`, `executors/v0.3.x`, `docs/contributing/howto/{committing,releasing}/*`	The repository version and executor references advance. Release documentation states that platform assets provide active executor lines.
Hash and executor guidance `.agents/skills/agentic-fuzzing/SKILL.md`, `docs/contributing/howto/testing/integration.md`	Testing guidance states that active executor lines track hash sidecars. Fuzzing guidance uses `stable_hash: false` for per-run comparisons.
v0.3 breaking-change documentation `docs/website/src/spec/changelog.rst`	The changelog adds v0.3 entries for memory overhead, loaded-runner limits, and VFS path limits.

CI summary handling

Layer / File(s)	Summary
Pre-existing change disclaimer `support/ci/pipelines/checks.py`, `support/ci/unit_tests/test_cargo_clippy.py`	Clippy summaries omit patch content when changes existed before the fix run. The test verifies the disclaimer and absence of the dirty file path and diff fence.

Priority: ⬇️ Low

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Merge Risk: ⚪ Minimal · up to c2cad

No established behavior in this change requires a fix before merging.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 8.70% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 23 functions across 8 files. (14 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the two primary changes: manager-side executor worker concurrency configuration and hardened post-install dependency handling.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 8.70% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 23 functions across 8 files. (14 skipped: 14 unsupported.)

Fix all pre-merge checks with AI

✨ Finishing Touches 💡 2

📝 Generate docstrings 💡

Create stacked PR
Commit on current branch

🛠️ Fix failing CI checks 💡

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/worker-control

Warning

Some tools did not complete. Review the errors below.

🔧 Clippy (1.98.0)

Clippy execution failed

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-09-17T09:49:29Z

Linked executor PR(s)

executor: genlayerlabs/genvm-executor#41 (v0.2)
executor: genlayerlabs/genvm-executor#42 (v0.3)

github-actions · 2026-09-17T09:49:54Z

GenVM PR actions

Tick a box to run it (the box unticks itself when handled). Actions only run while the PR has the ci-safe label.

Force run full tests
Provision executor PRs

Commands

/genvm-run-tests — run full tests once for the current manager snapshot
/merge — queue the exact manager snapshot through the App-owned E2E merge train

kp2pml30 · 2026-09-17T10:17:48Z

/genvm-run-tests

github-actions · 2026-09-17T10:18:04Z

👀 Full tests are running for c2caddc40424: open run #35209746458

kp2pml30 · 2026-09-17T11:29:20Z

/run-e2e

ci-core-e2e-runner · 2026-09-17T11:29:28Z

E2E status was updated. Follow the current E2E and merge checks on this PR. Detailed diagnostics are available internally.

kp2pml30 added 8 commits September 17, 2026 14:47

fix: drain executor logs and polish CI diagnostics 🐛

452db4f

feat(manager): configure executor worker concurrency ✨

5613ae6

chore(release): advance manager and executor release candidates 🚀

a53a21b

fix(manager): pass worker control in execution input 🐛

bb8bffa

chore(executor): pin worker ordering regression test ✅

a3068f5

chore(post-install): hash-pin venv deps and warn on unverified runner…

4e6e5b1

…s 🔒️

chore(post-install): drop the never-published executor download 🔥

776498a

fix(post-install): keep accepting --arch from forwarding callers 🐛

c2caddc

kp2pml30 self-assigned this Sep 17, 2026

kp2pml30 mentioned this pull request Sep 17, 2026

feat(manager): configure executor worker concurrency, harden post-install genlayerlabs/genvm-executor#41

Merged

kp2pml30 mentioned this pull request Sep 17, 2026

feat(manager): configure executor worker concurrency, harden post-install genlayerlabs/genvm-executor#42

Merged

github-actions Bot added the ci-safe label Sep 17, 2026

kp2pml30 added the test-release-pipeline label Sep 17, 2026

coderabbitai Bot approved these changes Sep 17, 2026

View reviewed changes

github-actions Bot added the run-full-tests label Sep 17, 2026

kp2pml30 merged commit bedbdd8 into v0.6-dev Sep 17, 2026
44 of 52 checks passed

kp2pml30 deleted the feat/worker-control branch September 17, 2026 13:28

Navigation Menu

Repository navigation

feat(manager): configure executor worker concurrency, harden post-install - #43

feat(manager): configure executor worker concurrency, harden post-install#43
kp2pml30 merged 8 commits into
v0.6-devfrom
feat/worker-control

kp2pml30 commented Sep 17, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Sep 17, 2026 •

edited

Loading

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

github-actions Bot commented Sep 17, 2026

Uh oh!

github-actions Bot commented Sep 17, 2026 •

edited by kp2pml30

Loading

Uh oh!

kp2pml30 commented Sep 17, 2026

Uh oh!

github-actions Bot commented Sep 17, 2026

Uh oh!

kp2pml30 commented Sep 17, 2026

Uh oh!

ci-core-e2e-runner Bot commented Sep 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

kp2pml30 commented Sep 17, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem and outcome

Implementation and validation

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

github-actions Bot commented Sep 17, 2026

Linked executor PR(s)

Uh oh!

github-actions Bot commented Sep 17, 2026 • edited by kp2pml30 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GenVM PR actions

Uh oh!

kp2pml30 commented Sep 17, 2026

Uh oh!

github-actions Bot commented Sep 17, 2026

Uh oh!

kp2pml30 commented Sep 17, 2026

Uh oh!

ci-core-e2e-runner Bot commented Sep 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

kp2pml30 commented Sep 17, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Sep 17, 2026 •

edited

Loading

github-actions Bot commented Sep 17, 2026 •

edited by kp2pml30

Loading