Repository navigation
fix(openshell): make the [openshell] extra enforce ACS policy again - #4166
Imran Siddique (imran-siddique) wants to merge 26 commits into
Conversation
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
…ll-acs-integration Signed-off-by: Imran Siddique <imran.siddique@opaque.co> # Conflicts: # .cspell-repo-terms.txt
The openshell skill's test fixture manifest declared `agent_control_specification_version: 0.3.0-alpha-agt`. After #3939 retargeted the policy engine onto the published agent-control-spec crate, the runtime refuses it: RuntimeError: runtime_error:manifest_invalid: unsupported agent_control_specification_version '0.3.0-alpha-agt'; supported versions are 0.4.0-alpha.1 This surfaced when main was merged into this branch, which had been 85 commits behind and so had not seen the retarget. Only the version string changes. `extends` and `tools`, which this fixture also uses, remain valid blocks in 0.4.0-alpha.1 per policy-engine/README.md's schema table, so #3940's fail-closed on removed manifest fields does not reach them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015QRxFm1Z1kE9iraPspwr7j Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
PR Review Summary
Verdict: AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims. |
📦 Dependency diff (SBOM)Comparing main → agent/restore-openshell-acs-integration. ✅ No dependency changes detected. |
|
🔴 Contributor Check: HIGH
Automated check by AGT Contributor Check. |
|
Two notes on 9cc02a2 while the restore-or-retire decision is pending. The autofix commit 808dc93 has no |
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
9cc02a2 to
96ccf70
Compare
|
Signed-off 808dc93 as 78933a9 and redid the main merge on top, with the tree unchanged. All 150 checks now pass on 3a4f49f, DCO included. The relay test passed this time with no agent-mesh change, so it is intermittent, as you said. I found no issue tracking it, and it was absent from the last 15 failed runs of that workflow, so it looks rare. The two open code-quality threads are false positives. |
Prayag (prayagupa)
left a comment
There was a problem hiding this comment.
Focused review comments on request binding, concurrent teardown, and package dependencies.
MohammadHaroonAbuomar
left a comment
There was a problem hiding this comment.
- prayagupa Major pyproject: openshell extra is empty though module imports ACS; declare compatible ACS package + clean wheel-install test
…ardown - Freeze the command before ACS evaluation and execute that frozen value, so a caller's list cannot change between approval and execution. - Keep the saved originals as an immutable mapping taken by each wrapper on entry, so a call in flight survives the last scope's restore. - Declare agent-control-specification on the [openshell] extra and check a clean wheel install of it in the openshell CI leg. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
All three items are fixed at ce8ee4e and verified by probe: the command is frozen to a tuple before the ACS snapshot and the same frozen value executes in all four wrappers and in the TRANSFORM branch (a second thread mutating the caller's list no longer changes what runs); the originals live in an immutable mapping and an in-flight call survives final-scope teardown with a single restore; the openshell extra declares the ACS range and the CI clean-install step exercises it. 35 tests pass, 22 of 22 checks green. Dismissing; one separate finding follows as a comment.
MohammadHaroonAbuomar
left a comment
There was a problem hiding this comment.
- prayagupa's three items are fixed at ce8ee4e and I verified each by probe: the command is frozen before the ACS snapshot and the same frozen tuple is what every wrapper and the TRANSFORM branch executes, so a second thread mutating the caller's list no longer changes what runs; the originals are an immutable mapping and an in-flight call survives final-scope teardown with exactly one restore; the
openshellextra declaresagent-control-specification>=0.4.0b0,<0.5.0, matching core, and the CI clean-install step installs and imports it from the local wheel. 35 tests pass, 22 of 22 checks green. One thing the probes turned up, pre-existing rather than introduced here, but in the function you just added, so worth folding in.
subprocess.Popen runs any non-string iterable as argv, but _freeze_command and _command_args only recognised list and tuple. A deque, UserList, dict.fromkeys(argv) or iterator was shlex-split from its repr, so ACS evaluated a nonsense executable while the real argv ran, and a deque stayed mutable after approval. Freeze every non-str, non-bytes, non-PathLike iterable into a tuple, which is what Popen would execute. A path-like cwd was snapshotted with os.fspath but the caller's object went on to execution. Resolve it once in the wrappers so the evaluated directory is the one used. Tests cover a policy that denies argv[0] == sys.executable for deque, UserList, dict keys and an iterator, a deque mutated after approval, and a PathLike cwd that changes on its second resolution. All six fail on ce8ee4e. Signed-off-by: Imran Siddique <imran.siddique@opaque.co> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Imran Siddique <imran.siddique@opaque.co> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixed at ee7f569 (head e2225fe): _freeze_command now treats any iterable other than str, bytes or PathLike as argv and freezes it to a tuple, so ACS evaluates the real command for a deque, a UserList, dict keys or an iterator, and a deque mutated after approval no longer changes what runs; cwd is resolved once with _freeze_cwd and the wrappers execute that string. Six new tests cover each case and the reply states all fail on the previous head. All 17 checks green. Dismissing; nothing of mine is open on this PR.
|
With Run at Splitting on the first token would not cover a chained command like |
| if isinstance(command, (list, tuple)): | ||
| return [_stringify_arg(arg) for arg in command] | ||
| text = _stringify_arg(command) | ||
| if shell: |
There was a problem hiding this comment.
agent-governance-python/agentmesh-integrations/openshell-skill/openshell_agentmesh/skill.py:304 Confirming the shell=True report on this head with the package's stub ACS and a rule that denies executable == 'rm': an argv list is blocked, but subprocess.run("rm f", shell=True), os.system("rm f"), os.popen("rm f") and "true; rm f" all run and delete the file, because the shell string is returned as a single argv element and line 90 reports its basename as the executable (f.txt, or rm f for /bin/rm f). A bytes command with shell=True behaves the same. Please report sh (or the platform shell) as the executable for shell strings and keep the full text in command, so shell policy keys on the command text, and add the four cases above as tests.
| # Evaluate and execute one immutable value: a caller's list could | ||
| # otherwise change between approval and execution. | ||
| command = _freeze_command(command) | ||
| args = _command_args(command, shell=shell) |
There was a problem hiding this comment.
agent-governance-python/agentmesh-integrations/openshell-skill/openshell_agentmesh/skill.py:86 A second bypass that does not need shell=True: subprocess.run(["x", f], executable="/bin/rm") runs rm and deletes the file while the policy sees executable x and argv ['x', f]. The wrappers ignore Popen's executable= keyword, which replaces the program that runs. Please feed executable= into the target (as the executable, with argv[0] kept) and add a test.
Replaces #3728, which carried six weeks of CI history and follow-up comments. Same branch, same head (
efa88590), nothing new in the code. This description is written for the review.The problem
pip install agent-governance-toolkit-integrations[openshell]installs anopenshell_agentmeshmodule that does nothing except emit a deprecation warning. The v5 migration removed the working adapter and never replaced it with an ACS one, so the extra we still advertise is empty.What this adds
GovernanceSkillandgoverned_shellare back, now backed by ACS. Everysubprocess.run,subprocess.Popen,os.systemandos.popeninside agoverned_shellscope goes throughsession.pre_tool_callasshell.executebefore the process starts. The policy-check CLI and the consolidated-package entry point come back with it.Where to look (15 minutes)
353 lines of production code in
skill.pyandcli.py, 625 lines of tests, the rest docs and CI wiring.skill.py:100: if policy evaluation throws, the command does not run. The broadexcept Exceptionis on purpose.skill.py:103: any verdict that does not permit (DENY, legacy ESCALATE) blocks before the subprocess starts.skill.py:105-111: a TRANSFORM with a malformed target raises instead of running the original command.skill.py:138-180: nested and concurrent scopes are tracked in aContextVar, and the four builtins are restored by reference even when the governed body raises.Validation
144 checks pass on
efa88590, 7 skipped, no conflicts. The native OpenShell CI job builds ACS and runs 33 tests. Removing the gate atskill.py:103makes three subprocess tests fail, which is how I know they test the gate and not the stub.Out of scope: v4 policy objects still need migrating to ACS, and this does not add a Vertex provider.