refactor(policy-engine)!: retarget onto the published agent-control-spec engine

[liamcrumm]

Summary

Replace AGT's embedded policy decision engine with agent-control-spec =0.4.0-alpha.3 and agent-hooks-sdk =0.1.0-alpha.5, the latest published pair verified September 8, while retaining AGT's existing host SDKs and native bindings. This is a breaking engine/host migration, not the six-package restructure.

Problem

AGT maintains a second implementation of an extracted engine. ACS now returns a three-decision verdict; AGT must apply transforms, handle enforcement mode and approvals, and retain its legacy host result shape.

The September 8 reassessment also found release and compatibility defects in this branch. The latest commits merge current main, repair those defects, and distinguish pinned-engine limitations from functionality added in later ACS releases.

Changes

Surface	Change
policy-engine/core	Compatibility aliases over agent-control-spec =0.4.0-alpha.3. Derive the legacy array-typed constant from the public supported-version list with a cardinality guard; retain behaviorally necessary bounded validation, telemetry and identity helpers.
Rust host and native bindings	Host-owned transforms, evaluate-only handling and identities, with full-snapshot validation. Explicit OPA backend selection survives Cargo feature unification. Public runtime getters replace duplicate construction state in the host and C ABI.
.NET	Move the C ABI to the Rust host SDK; preserve legacy host APIs; normalize missing approval infrastructure to host_error:approval_unresolved; version all five packages at 0.4.0-beta.0.
Python and tooling	Version the SDK at 0.4.0b0; require it from the generator and agt-policies 5.1.0. The consolidated core requires agt-policies>=5.1.0,<6.0.
Manifests, policies and examples	Migrate manifest version to 0.4.0-alpha.1, manifest root $policy_target to $target, and five-decision assumptions to the three-decision contract.
Release and CI	Build the renamed .NET native library for all five RIDs. Install local Python prerequisites before package, integration, container and fuzz consumers. Order ESRP PyPI jobs by dependency and retain release/supply-chain gates.
Shared test dependencies	Replace the incompatible typing-extensions 4.15.0 lock entry with aged, hash-verified 4.16.0. Add tooling-level pin validation and an agent-os runtime import regression.
Tests and documentation	Restore shared conformance coverage; exercise actual native artifacts and installed wheels; record migration steps, current upstream capabilities and outstanding release conditions.

All four direct ACS dependencies and all three consumer lockfiles use the updated registry pair. The crate checksum matches the registry and its trusted-publishing record. HTTP dependencies remain on aged ureq 3.4.0 / ureq-proto 0.6.1 rather than their September 6 successors.

Compatibility and rollout

• warn becomes allow with warnings[]; escalate becomes a liftable deny with approval.
• The manifest grammar does not accept $policy_target as an alias. The transform-path parser is a separate contract.
• Package alpha.3 still uses manifest grammar 0.4.0-alpha.1; manifests must not be bumped to the package version.
• Legacy hosts explicitly retain OPA semantics. Direct upstream AcsInterceptor / ActivatedPolicy consumers follow ACS's feature selection. Enabling Rego elsewhere cannot silently switch the legacy host or C ABI.
• Alpha.3 has a no-backend compilation defect. The core shim and telemetry crate explicitly enable the lightweight opa feature while keeping upstream defaults off, and are checked independently so workspace feature unification cannot hide this issue.
• Alpha.3's telemetry JSON helper does not preserve underscores in wire names. The correct compatibility projection stays until upstream behavior matches it. The newer artifact diagnostics likewise do not replace the legacy source-located OPA diagnostic API.
• Publish the Python SDK before its generator/migration-tool consumers, then release the consolidated core with the next repository-wide version bump.
• The .NET native build retains its existing opa,bundled-dispatchers feature selection. It requires trusted manifests and transitive configuration. The default Rust SDK and production Python/Node builds do not enable bundled annotators.
• ESRP orders SDK, policy tools, core, and dependent PyPI publications. GitHub actual PyPI publication is restricted to one selected package per run in dependency order; bulk dry-runs remain supported. This prevents the alternate parallel matrix from exposing consumers before prerequisites.
• Rust core and npm package versions still require coordinated release preparation. Do not republish modified code under existing 0.3.1 versions. Publish newly versioned prerequisite artifacts, then update exact dependent pins after the supply-chain gates accept them. No package publication is performed by this PR maintenance work.

See policy-engine/docs/acs-retarget.md for the symbol mapping and release order.

Outstanding merge condition

The selected alpha.3 artifact has repository metadata and a trusted-publishing record bound to upstream commit 4c47b57033b98c0d2ccf1b94624f058815db0a9c. The review's remaining organization/team co-owner condition is not waived: the registry still lists one individual owner. Upstream responsibleai/agent-control-spec#24 remains open.

The cooling-off scanner fix was split into #3564 and merged August 1. Its code is not changed here.

Pinned-engine limitations remain tracked upstream in responsibleai/agent-control-spec#20, #21, #22 and #23. The original binding-validation gap #14 was closed by #15; later releases expose more APIs, so the former blanket claim that bindings only expose AcsInterceptor is no longer current.

Testing

Local validation of the upstream upgrade:

• Rust workspace tests, host regressions, formatting and clippy with -D warnings.
• Cross-feature tests enable upstream Rego and prove that its default can evaluate without OPA while both legacy AGT host and C ABI retain OPA executable behavior. This test caught the initial C ABI backend leak before it was fixed.
• Isolated core --no-default-features --lib and otel --lib checks pass. The compatibility test also compiles the legacy supported-version constant as [&str; 1].
• Installed production Python wheel plus generator/migration tests: 547 passed, 36 skipped, 44 subtests passed. Installed core-to-policy-to-SDK dependency chain passes pip check and generates a valid manifest.
• Python source distribution builds an installable wheel.
• Node test build: 123 passed, one skipped. Default-feature production build also passes the full-snapshot regression.
• .NET solution build and 25 console test groups pass against the renamed native host library.
• C ABI artifact probe covers allow, deny, warnings, approvals, transforms, evaluate-only, malformed requests and full-snapshot overflow.
• 93 CI contract tests, generated-workflow consistency, version synchronization, source/crate packaging, cooling-off checks and changed-document link checks pass.
• Fuzz configuration retains the previously verified dated nightly compiler, target sanitizer instrumentation and Cargo coverage wrapper. The new CI run must validate the updated engine with that configuration.

Both independent reviewers confirmed their findings resolved with no new actionable issues at aa25d0af. The legacy constant type and isolated no-backend builds have regression coverage, and the generated CI checks pass.

The previous agent-os CI failure involved AnyIO 4.15.1 and the hashed test pin typing-extensions==4.15.0; it was also present in the merged main baseline. Commits f6b69289 and ed0f9bb3 fix that mismatch with 4.16.0 and regression coverage. All three agent-os jobs now pass.

Final verification at ed0f9bb3: 151 checks pass, none fail and none are pending. Five checks are skipped/neutral, including the separate CodeQL comparison warning explained below. The small shared-dependency delta was directly reviewed; the earlier engine upgrade has clean independent reviews. The registry ownership condition is not waived, and the active Protect ruleset still requires a code-owner approval of the latest push. This is not yet a merge-ready verdict.

CodeQL comparison warning

The missing default configuration is a separate Microsoft SDL API-upload analysis on main (48 rules, 54 findings), including custom rules not present in the standard Python/JavaScript workflow. It is not an empty duplicate and has not been deleted or replaced with a placeholder report. The current enforced required-status list does not include CodeQL; the existing workflow analyses are working. A matching SDL comparison needs the original publisher/query-pack access. The warning remains documented rather than hidden.

Attribution and AI assistance

Depends on the upstream engine extracted from this repository, responsibleai/agent-control-spec, and the responsibleai/agent-hooks contract. Written with GitHub Copilot CLI. No review comments or replies were posted as part of the September 8 maintenance.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[github-actions]

PR Review Summary

Check	Status	Details
🔍 Code Review	⚠️ Missing	No current-run comment
🛡️ Security Scan	⚠️ Missing	No current-run comment
🔄 Breaking Changes	⚠️ Missing	No current-run comment
📝 Docs Sync	⚠️ Missing	No current-run comment
🧪 Test Coverage	⚠️ Missing	No current-run comment

Verdict: ⚠️ AI review incomplete; ready for human review

AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims.

[github-actions]

Dependency Review

The following issues were found:

• ✅ 0 vulnerable package(s)
• ✅ 0 package(s) with incompatible licenses
• ✅ 0 package(s) with invalid SPDX license definitions
• ⚠️ 1 package(s) with unknown licenses.
• ⚠️ 2 packages with OpenSSF Scorecard issues.

View full job summary

[github-actions]

📦 Dependency diff (SBOM)

Comparing main → liamcrumm/retarget-acs-registry.

Summary: ➕ 5 added · ➖ 19 removed · 🔄 10 bumped

➕ Added

cargo (5)

Package	Version
agent-control-spec	0.4.0-alpha.3
agent-hooks-sdk	0.1.0-alpha.5
ryu-js	1.0.3
ureq-proto	0.6.1
utf8-zero	0.8.1

➖ Removed

cargo (19)

Package	Version
anes	0.1.6
cast	0.3.0
ciborium	0.2.2
ciborium-io	0.2.2
ciborium-ll	0.2.2
criterion	0.7.0
criterion-plot	0.6.0
crossbeam-deque	0.8.6
crossbeam-epoch	0.9.18
crossbeam-utils	0.8.21
crunchy	0.2.4
half	2.7.1
oorandom	11.1.5
plotters	0.3.7
plotters-backend	0.3.7
plotters-svg	0.3.7
rayon	1.12.0
rayon-core	1.13.0
tinytemplate	1.2.1

🔄 Bumped

cargo (3)

Package	From	To
napi-build	2.3.2	2.4.0
portable-atomic	1.13.1	1.14.0
unicode-segmentation	1.13.2	1.13.3

other (7)

Package	From	To
agent-control-specification-node	0.3.1-beta.0	0.4.0-beta.0
agent_control_specification_annotators	0.3.1-beta.0	0.4.0-beta.0
agent_control_specification_mcp	0.3.1-beta.0	0.4.0-beta.0
agent_control_specification_openai	0.3.1-beta.0	0.4.0-beta.0
agent_control_specification_otel	0.3.1-beta.0	0.4.0-beta.0
agent_control_specification_py	0.3.1-beta.0	0.4.0-beta.0
agent_control_specification_rig	0.3.1-beta.0	0.4.0-beta.0

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[Copilot]

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

[MohammadHaroonAbuomar]

Liam, this is next in line to land, but it's in merge conflict with main after today's merges. Could you rebase when you get a chance? I'll run the full re-review on the rebased head and land it; anything we find goes in a separate follow-up PR after yours is in.

[MohammadHaroonAbuomar]

Liam, to land this today I've carried your branch onto main in #3939 (your commits unchanged, two small conflict resolutions described there, co-authored trailer for attribution). It runs through the full review group now; anything found goes in a follow-up PR after it merges. I'll close this one once #3939 is in. Shout if you'd rather take it back.

[MohammadHaroonAbuomar]

Landed via #3939 with your commits and attribution intact; thanks Liam. The follow-up fixes from the group review are in #3940.