Skip to content

feat(acs): remote/pinned system prompt and rego bundle, plus top-level manifest URL loading - #3101

Merged
MohammadHaroonAbuomar merged 28 commits into
mainfrom
liamcrumm/acs-remote-prompt-and-bundle
Jun 22, 2026
Merged

MohammadHaroonAbuomar merged 28 commits into
mainfrom
liamcrumm/acs-remote-prompt-and-bundle

Conversation

@liamcrumm

@liamcrumm liamcrumm commented Jun 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Two additions to the bundled ACS dispatchers, both reusing the existing manifest extends HTTPS fetch path and its sha256/integrity trust gate. The bundled LLM annotator can now load its system prompt from a manifest-relative file or a pinned URL, and a rego policy can load its OPA bundle from a pinned URL. Both resolve at dispatch time and fail closed on any read, fetch, or hash error.

Problem

The bundled LLM annotator only accepted an inline system_prompt/prompt string, and the bundled OPA dispatcher only accepted a local bundle path. Operators who keep prompts or policy bundles in a central, integrity-pinned location had no first-class way to reference them. The core already does network I/O for URL extends with an HTTPS-only + hash-pin trust gate, so the safe machinery existed but was not reachable from the dispatchers.

Changes

File What changed
core/src/dispatchers/constants.rs Add system_prompt_file / system_prompt_url field name constants.
core/src/dispatchers/llm.rs resolve_system_prompt selects one of inline / file / pinned-URL; reads the file or fetches the URL at dispatch; missing file or fetch error fails closed as an annotator error.
core/src/manifest.rs Extend resolve_relative_paths to rewrite system_prompt_file like a rego bundle. Add validate_annotator_prompt_sources (at-most-one source, HTTPS+pin), the shared validate_pinned_https_url, and fetch_pinned_https_bytes/fetch_pinned_https_text reusing HttpExtendsFetcher + verify_extends_hash.
core/src/policy.rs Add RegoPolicyConfig.bundle_url and RegoPolicyInvocation.bundle_url; expose resolve_relative_string as pub(crate); reject bundle+bundle_url, unpinned, and non-HTTPS in validate_policy_definition.
core/src/opa.rs At dispatch, fetch a pinned bundle_url to a fresh private temp dir, pass the local path to opa eval --bundle, and remove the temp dir on completion. URLs are never shelled into opa.
spec/SPECIFICATION.md Document the three prompt sources (section 10) and bundle_url (section 12.1).
spec/schema/manifest.schema.json Add system_prompt_file, system_prompt_url, and rego bundle_url shapes (object with url + exactly one of sha256/integrity).
core/tests/opa.rs Update rego_invocation test helper for the new field.

Design notes

  • Dispatch-time resolution, single validation point. Resolution happens in the dispatchers (where the core already does I/O), not at load time. Manifest::validate checks the raw fields, so the at-most-one-source and HTTPS+pin rules are enforced for both file-based loading and from_native construction (the runtime constructor calls validate). Load-time inlining was rejected because validate runs after resolution and could no longer see an illegal combination.
  • Stricter than extends. URL extends permits an unpinned string form; a system_prompt_url / bundle_url MUST carry a pin. An unpinned or non-HTTPS URL is rejected.
  • Known tradeoff. Both new URL paths re-fetch per evaluation (the runtime is stateless). Integrity is guaranteed by the pin and bodies are capped at the URL-extends limit (1 MiB). A content-addressed cache could remove the re-fetch in a follow-up; it is intentionally out of scope here.

Testing

cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
AGENT_CONTROL_REQUIRE_OPA=1 cargo test --workspace   # opa 0.70.0 on PATH

All three pass. New unit tests cover: prompt-source mutual exclusion, HTTPS+pin validation for both fields, the pinned-fetch trust gate via a mock fetcher (sha256 match, SRI integrity, hash mismatch, missing pin not fetched), system_prompt_file read end-to-end through dispatch, fail-closed on a missing file, bundle+bundle_url rejection, and the remote-bundle temp-dir lifecycle (materialize then clean up on drop).


Added: load the top-level manifest from an HTTPS URL

Summary

A loader can now fetch the top-level manifest itself from an HTTPS URL, not just local extends. Manifest::from_url(url, sha256?) reuses the same URL-extends trust gate (https-only, no ambient credentials, bounded body, sha256 verify). The pin is optional, matching URL extends: an unpinned URL is trusted because the host chose it. When a sha256 is supplied it is verified and a mismatch fails closed.

Problem

Today a manifest can only be loaded from a URL indirectly, by writing a local stub whose extends points at the URL. Hosts that store the whole policy centrally (e.g. a Foundry SDK integration pointing at https://policies.example/governance.acs.yaml) had no first-class from_url.

Changes

File What changed
core/src/manifest.rs Add Manifest::from_url / from_url_with_limits (sha256: Option<&str>) and ManifestLoader::load_url, reusing validate_https_url, validate_extends_trust, fetch_url_body, verify_extends_hash, and ManifestLocation::Url. A URL-sourced manifest resolves its own extends against the URL (relative refs become sibling URLs) and never reaches the local filesystem.
spec/SPECIFICATION.md New section 2.3 (loading from a URL) and a note in section 1.1.
core/src/ffi.rs acs_builder_from_url(url, sha256, err).
sdk/rust/src/host/mod.rs AgentControl::from_url / from_url_with_dispatchers.
sdk/python/src/lib.rs PyO3 NativeRuntime.from_url.
sdk/python/agent_control_specification/_client.py, _orchestration.py NativeRuntimeClient.from_url and AgentControl.from_url wrappers.
sdk/node/native/lib.rs, sdk/node/src/index.ts napi fromUrl factory + TS facade AgentControl.fromUrl.

Design notes

  • Pin optional, mirroring extends. An unpinned URL is trusted because the host chose it (same rule as URL extends). When a sha256 is supplied it is verified; a mismatch, a malformed pin, a non-https URL, a fetch error, or a body-size breach fails closed. An empty/whitespace pin normalizes to no pin.
  • sha256-only surface. The optional pin is a sha256 hex string (the SRI/integrity form remains an additive change) to keep one uniform signature across all binding layers.
  • URL-sourced manifests are untrusted for host-local access. A manifest loaded via from_url is flagged url_sourced and rejected at load if it declares any filesystem path field (bundle, system_prompt_file, cedar policy_path/entities_path/schema_path, adapter data/data_paths) or a remote rego bundle_url. It references a remote prompt via system_prompt_url or supplies policy and credentials inline. See the security hardening section below for the full trust model.

Testing

New tests: core (MockFetcher) for pinned fetch+merge, unpinned-allowed (missing/whitespace pin), sha256 mismatch, non-https, body-size limit, pinned remote extends, and relative-ref-resolves-to-URL; Rust host and Python layers for non-https rejection and the optional-pin call path. Python binding verified end-to-end via maturin develop (_native.NativeRuntime.from_url present; unpinned from_url reaches the loader; http:// fails closed). All ACS gates green: cargo fmt --all -- --check, cargo clippy --workspace --all-targets -- -D warnings, AGENT_CONTROL_REQUIRE_OPA=1 cargo test --workspace (opa 0.70.0).


Added: security hardening for URL-sourced manifests

Summary

from_url lets a host load a manifest it does not author, so a URL-sourced manifest is treated as untrusted. Several guards keep an untrusted remote manifest from reaching host-local files, host credentials, or internal network destinations. These address the review findings from MohammadHaroonAbuomar.

Guards

Guard Behavior
Local-access rejection A url_sourced manifest is rejected at load if it declares a filesystem path field or a remote rego bundle_url. The scan runs on the fully extends-merged manifest and on each annotator declaration overlaid with its intervention-point binding.
Credential suppression A url_sourced manifest also controls an llm annotator endpoint, so the bundled LLM dispatcher never reads a host environment credential for it, including provider default variables (OPENAI_API_KEY, AZURE_OPENAI_API_KEY, GEMINI_API_KEY, AWS_SESSION_TOKEN). Credentials must be supplied inline.
Remote rego rejection A url_sourced manifest cannot carry bundle_url, because the bundled OPA dispatcher runs the fetched rego with host environment and network access (opa.runtime + http.send). bundle_url stays available to file-sourced, operator-authored manifests.
SSRF IP block Every URL fetch (from_url, extends, system_prompt_url, bundle_url) rejects loopback, link-local, unspecified, and broadcast IP literals, including the cloud metadata endpoint 169.254.169.254 and IPv4-mapped IPv6 forms. RFC1918 stays allowed for internal HTTPS hosting.
Redirect re-validation The fetcher follows redirects itself and re-runs the HTTPS and SSRF IP checks on each hop, so a vetted public URL cannot bounce to a loopback, metadata, or internal HTTPS host, or downgrade to HTTP. The hop count is capped.
Private temp dir A remote rego bundle materializes to a 0700 temp dir on Unix, removed on completion.
Host fetch limits The FFI builder (acs_builder_set_url_fetch_limits), Rust SDK (from_url_with_limits), Python (max_url_bytes / url_timeout_ms / max_url_redirects), and Node (urlFetchLimits) let a host tighten the body-size, timeout, and redirect caps used for dispatch-time fetches.

Residual scope

Hostname-resolved SSRF and DNS rebinding are not revalidated by the runtime and remain host network responsibilities. The per-evaluation re-fetch (no cache) is intentional so a revoked pin is picked up promptly; a content-addressed cache is a follow-up.


Added: real Azure AI Foundry Agents integration example

Summary

A genuine, non-mocked reference at policy-engine/sdk/python/examples/real_packages/foundry_agents.py showing how a production user governs Azure AI Foundry Agents tool calls with ACS. It builds real azure-ai-agents FunctionTool definitions and backs the policy with a live Azure OpenAI LLM judge (no canned verdicts), gated on real credentials via _common.require_azure.

Changes

File What changed
sdk/python/examples/real_packages/foundry_agents.py New example. Short path (control.protect_tool) and long path (control.evaluate_intervention_point with an allow/deny/escalate/transform switch). References from_path/from_url and system_prompt_file/system_prompt_url as production options.
sdk/python/examples/real_packages/README.md New. How to run the real-package examples (env + extra) and the Foundry walkthrough.
sdk/python/pyproject.toml Add azure-ai-agents>=1.1,<2 to the optional realpkg-tests extra.
scripts/check_dependency_confusion.py Allowlist azure-ai-agents (real Microsoft PyPI package) for the strict dependency-confusion scan.

Notes

  • Both code paths, one seam. Short path returns a drop-in async wrapper that evaluates PRE_TOOL_CALL/POST_TOOL_CALL, applies a transform, and raises AgentControlBlocked on deny. Long path is the explicit form for wiring ACS into a framework's own auto-function-call hook.
  • Tested invariant, not the model. The hard assertion is the security invariant ("a destructive call is never executed"), which holds for both a real judge deny and a fail-closed transient. A small retry-on-transient (retries only annotation_failed/annotation_timeout, never a real deny) keeps the live judge from flaking; verified 6/6 clean runs against a real gpt-5.x deployment.
  • No secret on disk. The manifest is assembled in-process so the Azure endpoint comes from the environment and the API key is referenced by name (api_key_env).

Also the ci tests were failing unrelated to this change so I just went ahead and fixed it (k12 tests)

liamcrumm and others added 2 commits June 17, 2026 20:10
The bundled LLM annotator preset previously read its system prompt only
from an inline `system_prompt` (or `prompt`) field. Add two more sources
that resolve at dispatch time and fail closed on any read or fetch error.

- `system_prompt_file`: a manifest relative path rewritten to absolute in
  `Manifest::resolve_relative_paths` (mirroring the rego `bundle` path
  rule) and read by the dispatcher at evaluation time.
- `system_prompt_url`: a pinned `{url, sha256|integrity}` object fetched
  over the existing extends fetch path and trust gate (HTTPS only, hash
  pin required, reusing `HttpExtendsFetcher` and `verify_extends_hash`).
  Unlike extends, an unpinned prompt URL is rejected.

`Manifest::validate` enforces that at most one prompt source is set and
that a `system_prompt_url` is HTTPS and pinned. Validation runs on raw
fields so it covers both file based loading and `from_native`
construction (the runtime constructor calls `validate`).

Updates SPECIFICATION.md section 10 and the manifest JSON schema, and
adds unit tests for validation, the pinned fetch trust gate (mock
fetcher), the file read path, and fail closed behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
The bundled OPA dispatcher previously evaluated rego only from a local
`bundle` path. Add a `bundle_url` that lets a rego policy reference a
remote bundle pinned by `sha256` or `integrity`.

- `RegoPolicyConfig.bundle_url` is a `{url, sha256|integrity}` object,
  mutually exclusive with `bundle`. `validate_policy_definition` rejects
  declaring both, an unpinned URL, and a non HTTPS URL, reusing the
  shared `validate_pinned_https_url` trust gate.
- At dispatch time `OpaRegoRunner` fetches the bundle over the extends
  fetch path (HTTPS only, hash verified), writes it to a fresh private
  temp directory, passes the local path to `opa eval --bundle`, and
  removes the temp directory when evaluation finishes. A fetch error,
  size breach, or hash mismatch fails closed before opa runs. URLs are
  never shelled into opa directly.

The fetched body inherits the URL extends byte cap. Updates
SPECIFICATION.md section 12.1 and the manifest JSON schema, and adds
validation tests plus a temp bundle lifecycle test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
@github-actions github-actions Bot added documentation Improvements or additions to documentation tests size/XL Extra large PR (500+ lines) labels Jun 17, 2026
@github-actions

github-actions Bot commented Jun 17, 2026 •

Copy link
Copy Markdown
🤖 AI Agent: code-reviewer — Action items:

AI-generated review output. Treat it as untrusted analysis and verify before acting.

TL;DR: 0 blockers, 2 warnings. Changes are well-implemented but require further review for edge cases and potential optimizations.

# Sev Issue Where
1 Warn Potential SSRF bypass via hostname-resolved SSRF or DNS rebinding not revalidated. core/src/manifest.rs, core/src/dispatchers/llm.rs
2 Warn Lack of caching for re-fetched resources may lead to performance bottlenecks. core/src/manifest.rs, core/src/opa.rs

Action items:

  • None (no blockers).

Warnings:

# Action Notes
1 Fine as follow-up PRs. Consider implementing hostname-resolved SSRF and DNS rebinding protections to address residual risks.
2 Fine as follow-up PRs. Explore adding a content-addressed cache for fetched resources to improve performance.

@github-actions

github-actions Bot commented Jun 17, 2026 •

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@github-actions

github-actions Bot commented Jun 17, 2026 •

Copy link
Copy Markdown
🤖 AI Agent: breaking-change-detector — API Compatibility

AI-generated review output. Treat it as untrusted analysis and verify before acting.

API Compatibility

Severity Change Impact
High Added url_sourced field to LlmConfig and modified its behavior to restrict access to host environment credentials when url_sourced is true. This change may break existing integrations that rely on environment credentials for LLM annotators if the manifest is marked as url_sourced.
High Introduced DefaultAnnotatorDispatcher with with_limits_and_source constructor, replacing the previous default constructor. Existing code that uses the default constructor for DefaultAnnotatorDispatcher will break, as it has been removed.
High Modified LlmConfig::from_fields to require additional parameters: limits and url_sourced. Any existing calls to LlmConfig::from_fields without these parameters will fail to compile.
High Changed dispatch_with_transport function signature to include limits and url_sourced parameters. Existing calls to dispatch_with_transport without these parameters will fail to compile.
High Added resolve_system_prompt function, altering the resolution logic for system_prompt, system_prompt_file, and system_prompt_url. Changes the behavior of system prompt resolution, potentially breaking existing configurations that rely on the previous resolution logic.
Medium Introduced stricter validation for system_prompt_url and bundle_url fields, requiring HTTPS and integrity pins. Existing manifests with non-HTTPS or unpinned URLs for these fields will now fail validation.
Medium Added Manifest::from_url and related APIs, which enforce stricter security checks for URL-sourced manifests. Existing workflows that attempt to load manifests from URLs without meeting the new security requirements may fail.
Medium URL-sourced manifests are now restricted from accessing host-local files, credentials, and certain network destinations. This may break existing use cases where URL-sourced manifests relied on these capabilities.

@github-actions

github-actions Bot commented Jun 17, 2026 •

Copy link
Copy Markdown
🤖 AI Agent: security-scanner — View details

AI-generated review output. Treat it as untrusted analysis and verify before acting.

No security issues found.

@github-actions

github-actions Bot commented Jun 17, 2026 •

Copy link
Copy Markdown

PR Review Summary

Check Status Details
🔍 Code Review ⚠️ Missing No current-run comment
🛡️ Security Scan ⚠️ Missing No current-run comment
🔄 Breaking Changes ⚠️ Missing No current-run comment
📝 Docs Sync ⚠️ Missing No current-run comment
🧪 Test Coverage ⚠️ Missing No current-run comment

Verdict: ⚠️ AI review incomplete; ready for human review

AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims.

liamcrumm and others added 5 commits June 17, 2026 22:33
The generator ships a copy of spec/schema/manifest.schema.json and
test_packaged_schemas_match_canonical_spec_schemas asserts they are
byte-identical. Propagate the system_prompt_file/url and bundle_url
additions into the packaged copy.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
…bundle errors

Deep review (repro-gated) found a validation gap: validate_annotator_prompt_sources
only checked the annotator declaration, but AnnotatorInvocation::from_annotation
merges the intervention point's annotation binding over the declaration at
dispatch. A binding could set an inline `prompt` (or system_prompt_file) that
silently overrode a pinned `system_prompt_url` on the declaration, defeating the
pin requirement.

- Refactor the prompt-source check into validate_prompt_source_fields and run it
  on the effective merged (declaration + binding) field set per opted-in
  annotation, so more than one source fails closed with manifest_invalid.
  Regression tests cover the binding-override-pinned-url case and the still-valid
  single-binding-source case.
- opa.rs: a dispatch-time `bundle_url` fetch/hash/non-https failure now fails
  closed as `policy_invocation_failed` rather than inheriting the extends path's
  `manifest_invalid`; a size breach keeps `resource_limit_exceeded`. This labels
  a runtime remote-fetch failure correctly for audit.
- Spec: drop the inaccurate "private" claim for the bundle temp dir (it is a
  dedicated dir removed after evaluation, not 0700), and state that a
  `system_prompt_file`, like a rego `bundle`, is not confined to the manifest
  directory. Document that the at-most-one-source rule counts the merged binding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
Add Manifest::from_url / from_url_with_limits, which fetch the top level
manifest from an HTTPS URL through the existing URL extends trust gate
(https-only, no ambient credentials, bounded body size, sha256/integrity
verify). Unlike an extends entry the pin is mandatory because the top
level manifest is the root of trust, so an unpinned remote root fails
closed. A URL sourced manifest resolves its own extends against the URL
and never reaches the local filesystem.

Document the behaviour in SPECIFICATION.md section 2.3 and note that
filesystem-relative fields are not rebased for URL manifests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
Thread the pinned-URL manifest loader through every binding that already
exposes from_path: the C FFI (acs_builder_from_url), the Rust host
(AgentControl::from_url), the Python PyO3 native runtime plus the
NativeRuntimeClient and AgentControl wrappers, and the Node napi factory
plus its TypeScript facade. Each is a thin pass-through to
Manifest::from_url and requires url + sha256.

Add fail-closed regression tests at the Rust host and Python layers
covering non-https rejection and the mandatory pin. Core fetch/verify
paths are covered by the MockFetcher tests added with the core change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
@liamcrumm liamcrumm changed the title feat(acs): remote/pinned system prompt and rego bundle for bundled dispatchers feat(acs): remote/pinned system prompt, rego bundle, and top-level manifest URL loading Jun 19, 2026
Align top-level from_url with the URL extends trust model, where an
unpinned URL is trusted because the host chose it. The sha256 pin is now
optional across every layer (Manifest::from_url takes Option<&str>;
Python sha256=None, Node sha256?, FFI accepts a null sha256). An empty or
whitespace pin normalizes to no pin. When a pin is supplied it is still
verified and a mismatch fails closed; HTTPS-only is still enforced with
or without a pin.

Update SPECIFICATION.md section 2.3 and the core/Rust-host/Python tests
accordingly (missing-pin is now an allowed happy path).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
@liamcrumm liamcrumm changed the title feat(acs): remote/pinned system prompt, rego bundle, and top-level manifest URL loading feat(acs): remote/optional-pinned system prompt, rego bundle, and top-level manifest URL loading Jun 19, 2026
@liamcrumm liamcrumm changed the title feat(acs): remote/optional-pinned system prompt, rego bundle, and top-level manifest URL loading feat(acs): remote/pinned system prompt and rego bundle, plus top-level manifest URL loading Jun 19, 2026
Add examples/real_packages/foundry_agents.py: a genuine, non-mocked
reference showing how a production user governs Foundry tool calls with
ACS. It builds real azure-ai-agents FunctionTool definitions and backs the
policy with a live Azure OpenAI LLM judge (no canned verdicts), gated on
real credentials via _common.require_azure.

It demonstrates both integration styles for the same governed seam: the
short path (control.protect_tool) and the long path (explicit
evaluate_intervention_point with an allow/deny/escalate/transform switch),
and points at from_path / from_url manifest loading and system_prompt_file
/ system_prompt_url for production. A retry-on-transient helper keeps the
live judge from flaking the run while still honoring real denies.

Wire azure-ai-agents>=1.1,<2 into the realpkg-tests extra and add a README
for the real-package examples directory.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
@github-actions github-actions Bot added the dependencies Pull requests that update a dependency file label Jun 19, 2026
@github-actions

github-actions Bot commented Jun 19, 2026 •

Copy link
Copy Markdown

📦 Dependency diff (SBOM)

Comparing main → liamcrumm/acs-remote-prompt-and-bundle.

✅ No dependency changes detected.

Comment thread policy-engine/sdk/python/examples/real_packages/foundry_agents.py Fixed
liamcrumm and others added 3 commits June 19, 2026 01:29
azure-ai-agents is the real Microsoft Azure AI Foundry Agents SDK on
PyPI, added to the realpkg-tests extra for the foundry_agents example.
Register it (both hyphen and underscore forms) so the strict
dependency-confusion scan recognizes it.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
Address the github-code-quality bot finding: the retry helpers mixed an
explicit return with an implicit fall-through return None. Restructure
both govern() and the short-path call() as a retry loop plus an explicit
final attempt so every path returns or raises explicitly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
… blank pin

Deep-review findings on the from_url loader:

- Security (reproduced): a URL sourced manifest could reference local files
  via a rego bundle, an annotator system_prompt_file, a cedar path, or an
  adapter data path. Those are not rebased for a URL manifest, so they
  resolved against the process working directory at dispatch and a remote
  (optionally unpinned) manifest could read a local file and exfiltrate it
  through a dispatcher. Add Manifest::reject_filesystem_path_fields, called
  from load_url, so every such field now fails closed. The spec section 2.3
  no longer overclaims and section 1.1 drops the stale 'pinned' wording.
- A supplied but blank sha256 silently became unpinned, unlike URL extends.
  Stop swallowing it so a present blank pin fails closed; only None is unpinned.
- Restore trust_root on every load_url path by taking it only around
  load_location_with_body, not across the fetch/verify early returns.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
@github-actions

github-actions Bot commented Jun 19, 2026 •

Copy link
Copy Markdown
🤖 AI Agent: docs-sync-checker — Docs Sync

AI-generated review output. Treat it as untrusted analysis and verify before acting.

Docs Sync

  • DefaultAnnotatorDispatcher in core/src/dispatchers/default.rs -- missing docstring for the with_limits_and_source method.
  • LlmAnnotator in core/src/dispatchers/llm.rs -- missing docstring for the with_limits and with_url_sourced methods.
  • resolve_system_prompt() in core/src/dispatchers/llm.rs -- missing docstring.
  • resolve_system_prompt behavior is not documented in spec/SPECIFICATION.md.
  • README.md -- no updates to reflect the new features (system_prompt_file, system_prompt_url, bundle_url, from_url).
  • CHANGELOG.md -- missing entry for new features (system_prompt_file, system_prompt_url, bundle_url, from_url) and security hardening changes.

@github-actions

github-actions Bot commented Jun 19, 2026 •

Copy link
Copy Markdown
🤖 AI Agent: test-generator — `policy-engine/core/src/dispatchers/default.rs`

AI-generated review output. Treat it as untrusted analysis and verify before acting.

policy-engine/core/src/dispatchers/default.rs

  • test_dispatcher_with_invalid_url_sourced_provenance -- Test that DefaultAnnotatorDispatcher correctly handles invalid url_sourced provenance.
  • test_dispatcher_with_invalid_limits -- Test behavior when invalid Limits are passed to DefaultAnnotatorDispatcher.

policy-engine/core/src/dispatchers/llm.rs

  • test_resolve_system_prompt_invalid_url -- Test resolve_system_prompt with an invalid or non-HTTPS system_prompt_url.
  • test_resolve_system_prompt_missing_file -- Test resolve_system_prompt with a missing or unreadable system_prompt_file.
  • test_llm_config_url_sourced_credential_rejection -- Validate that LlmConfig rejects host environment credentials when url_sourced is true.

@MohammadHaroonAbuomar MohammadHaroonAbuomar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security review

Nice, careful PR — pin-required for system_prompt_url/bundle_url, blank-pin fails closed, the binding-override bypass is closed, URLs are never shelled into opa, and everything fails closed on read/fetch/hash/size errors. A few concerns worth addressing, highest-impact first (inline comments anchor each one).

1. Remote (from_url) manifests can still exfiltrate process secrets (High). reject_filesystem_path_fields closes the filesystem half of its stated threat ("a remote manifest reads local secrets and exfiltrates them through a dispatcher") but not the environment half: a URL-sourced manifest fully controls an LLM annotator's endpoint + api_key_env, and the dispatcher does std::env::var(api_key_env) (llm.rs:219) and sends it as an auth header to endpoint. A malicious/compromised remote manifest can name any var (AWS_SECRET_ACCESS_KEY, …) and POST it to an attacker URL. See inline on reject_filesystem_path_fields.

2. No SSRF egress controls on any URL fetch (Moderate). The gate enforces https + no-creds + no-fragment but not destination: loopback, RFC1918, and link-local 169.254.169.254 (cloud metadata) are all reachable. Pinned fetches still fire the request before hash verification (blind SSRF / internal probing / GET side effects); unpinned from_url/extends allow full content-reflection SSRF; and followed redirects are not re-validated for host (only https_only), so a vetted public URL can bounce to an internal endpoint. See inline on validate_pinned_https_url.

3. Per-evaluation re-fetch → DoS amplification + availability coupling (Moderate). system_prompt_url and bundle_url resolve on every dispatch with no cache, mirroring request volume onto the pinned host and blocking every eval to timeout (then fail-closed deny) if it's slow/down. See inline on fetch_remote_bundle.

4. Dispatch-time fetches hard-code Limits::default() (Low–Moderate). Host-configured max_manifest_url_bytes / timeout / redirect limits are silently ignored for prompt and bundle fetches. See inline on the prompt fetch.

5. OPA temp dir isn't actually "private" (Low). create_dir under env::temp_dir() inherits the process umask (typically 0755/0644 → world-readable bundle on multi-user hosts); the predictable name also allows a local fail-closed DoS. See inline on write_remote_bundle.

6. Decompression-bomb surface for bundle_url (Low / informational). 1 MiB gzip → opa eval --bundle can expand large; mitigated by the pin (operator-chosen content), noted only.

Suggested priority: address #1 and #2 before merge; #3–#5 can be fast-follows. Findings 1–3 share one root cause — the gate authenticates content (pin) and scheme (https) but not destination (host) or provenance (who authored the manifest) — so a host-allowlist + egress policy at the gate would mitigate several at once.

Comment thread policy-engine/core/src/manifest.rs Outdated
Comment thread policy-engine/core/src/manifest.rs
Comment thread policy-engine/core/src/dispatchers/llm.rs Outdated
Comment thread policy-engine/core/src/opa.rs Outdated
Comment thread policy-engine/core/src/opa.rs

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security and correctness review: APPROVE.

URL fetch paths: HTTPS-only enforced at two layers (validate_extends_trust + validate_https_url), hash verification via verify_extends_hash, body capped at 1 MiB by the existing HttpExtendsFetcher, no credential headers. Fail-closed on any error.

OPA temp dir: No shell injection -- Command::new + .arg() throughout, never a shell string. Unique dir names ({pid}-{nanos}-{counter}). RemoteBundle::drop calls remove_dir_all; write-error path also cleans up before returning.

system_prompt_url at-most-one-source: validate_prompt_source_fields counts sources and rejects >1. validate_merged_annotation_prompt_sources extends this to the merged binding+declaration, so a binding prompt cannot silently override a pinned declaration URL. The regression test binding_prompt_cannot_override_pinned_declaration_url covers this exactly.

Manifest::from_url: Pin-optional for top-level URL load mirrors the extends trust gate (host chose the URL). reject_filesystem_path_fields blocks all local file access from URL-sourced manifests. Relative extends inside a URL-sourced manifest resolve against the URL base, not the filesystem. Trust root swap correct.

FFI/PyO3/napi: ffi_guard! prevents panics through FFI. Standard error propagation in all three SDK layers.

edu-k12.yaml regex fixes: All three fixes are correct and match the failing test inputs. This makes #3115 redundant -- will close it after merge.

dep-confusion allowlist: azure-ai-agents is a real Microsoft PyPI package; added correctly alongside the existing Azure SDK entries.

…PA temp dir permissions

Addresses two findings from @MohammadHaroonAbuomar's review:

Finding 4 (Low-Mod): dispatch-time remote bundle and system_prompt_url
fetches used Limits::default() instead of host-configured limits.
OpaRegoRunner and LlmAnnotator now each carry a limits field (defaults to
Limits::default() for zero-config callers) with a with_limits() builder
so the host can propagate its configured limits to both dispatchers.

Finding 5 (Low): the OPA temp directory was created with std::fs::create_dir,
which inherits the process umask. On multi-user Unix hosts this could leave
the bundle world-readable. Replaced with a create_private_dir helper that
uses DirBuilder::mode(0o700) on Unix (Windows is unchanged; ACLs apply).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Imran Siddique <imran.siddique@opaque.co>
@imran-siddique

Copy link
Copy Markdown
Collaborator

MohammadHaroonAbuomar addressed your review findings -- replied inline on all five threads. Findings 4 and 5 are fixed in 99f3a9a; 1, 2, and 3 are pre-existing or documented v1 limitations with follow-up tracking. Would you be able to resolve your threads and change the review to Approved? That's the last gate before merge.

liamcrumm and others added 5 commits June 19, 2026 21:21
Address @MohammadHaroonAbuomar's review of #3101:

- Finding 1 (High): a URL-sourced manifest also controls an llm annotator's
  endpoint, so my earlier filesystem-field guard only closed half the exfil.
  A remote manifest could still name api_key_env / aws_*_env and ship a host
  secret to a chosen endpoint, and the from_url pin is optional so the prior
  'the pin closes it' premise does not hold. Now reject host-env secret
  annotator fields on URL-sourced manifests; credentials must be inline.
- Finding 2 (Moderate, SSRF): reject loopback and link-local IP destinations
  at the URL trust gate (validate_url_components), blocking fetches aimed at
  the host itself or cloud metadata (169.254.169.254). RFC1918 stays allowed
  for internal hosting; hostname-resolved SSRF, DNS rebinding, and per-redirect
  re-validation are documented residual follow-ups.

Update spec sections 2.2 and 2.3 and add tests for both.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
Finding 4 follow-up: the with_limits() builders added in 99f3a9a were not
reachable from the zero-config default dispatchers, so an operator's tightened
limits were still ignored. Thread Limits end to end: DefaultAnnotatorDispatcher
now carries limits and builds the llm annotator with them, and new
default_annotator_dispatcher_with_limits / default_policy_dispatcher_with_limits
factories wire OpaRegoRunner::with_limits and LlmAnnotator::with_limits. The
existing zero-config factories delegate with default limits, preserving
behavior. Also fixes the cargo fmt break in 99f3a9a (opa.rs).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
The Finding 4 wiring put default_annotator_dispatcher_with_limits (not opa
gated) alongside a Limits import that was gated on the opa feature, so a build
with default-dispatchers but without opa (for example
--no-default-features --features openai_moderation) failed to compile with
'cannot find type Limits'. The default opa+cedar build hid it. Move Limits to
an unconditional import; it is a general type used by both the annotator and
the opa policy factories.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
…dings; fix IPv6 SSRF bypass

A second deep review of #3101 found the first env-exfil fix was incomplete and
the SSRF guard was bypassable. Three issues, all with regression tests:

1. Default-env credential exfil (High). Rejecting the api_key_env / aws_*_env
   fields did not stop the bundled llm dispatcher from falling back to a
   provider default credential (OPENAI_API_KEY, AZURE_OPENAI_API_KEY,
   GEMINI_API_KEY, and the bedrock AWS_SESSION_TOKEN sent verbatim) and shipping
   it to the manifest controlled endpoint. The env var name is a hardcoded
   constant, not a manifest field, so a field scan cannot see it. Mark a URL
   loaded manifest url_sourced and thread it to the llm dispatcher so it never
   reads a host environment credential (explicit or default); credentials must
   be inline. Provider agnostic, so it fails closed for future providers too.

2. Binding bypass. The host-secret and system_prompt_file rejection scanned only
   annotator declarations, but AnnotatorInvocation::from_annotation overlays
   intervention point binding fields, so a binding could inject api_key_env or
   system_prompt_file past a clean declaration. Now scan each declaration merged
   with its binding as well.

3. SSRF IPv4-mapped IPv6 bypass. is_blocked_fetch_ip missed [::ffff:169.254.169.254]
   and [::ffff:127.0.0.1] because Ipv6Addr::is_loopback/link_local are false for
   v4-mapped addresses. Canonicalize via to_ipv4_mapped/to_ipv4 before the check.

Update SPECIFICATION.md 2.2 and 2.3.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
@imran-siddique

Copy link
Copy Markdown
Collaborator

MohammadHaroonAbuomar -- pinging again on this one. Findings 4 and 5 from your review are fixed. All five threads have responses. Waiting on your approval to unblock the merge.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ACS dispatch work is well-engineered. Specific things I reviewed:

Security model for URL-sourced manifests: The url_sourced flag correctly prevents host env credential reads when the manifest itself was fetched from a URL. Without it, a compromised URL manifest could use api_key_env to read a host secret and exfiltrate it to an endpoint it also controls. The guard at resolve_api_key is the right place.

Fetch trust gate: system_prompt_url and bundle_url both go through fetch_pinned_https_text / fetch_pinned_https_bytes, reusing the extends hash-pin mechanism. Stricter than extends (pin is required, not optional) -- that's correct given these are dispatch-time fetches.

Known limitation on system_prompt_file in URL-sourced manifests: The PR body explicitly documents that filesystem-relative fields in a URL-sourced manifest are not rejected, mirroring URL-extends behavior. That's an acceptable deferred scope given the doc note in the spec. Worth adding a validator rejection in a follow-up.

OPA bundle_url temp dir lifecycle: Materializing to a private temp dir and dropping it on completion is safe. URLs are never shelled into opa.

One concern to coordinate: This PR includes three edu-k12.yaml regex fixes that overlap with PR #3115. The changes in this PR look correct, but merging this before #3115 will cause a conflict there (or make its fixes redundant). Please coordinate with the #3115 author -- either cherry-pick from one PR to the other, or close #3115 if these fixes are a superset.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACS feature is well-implemented with correct security layering.

Security model: two enforcement points

  1. Load-time: reject_url_sourced_local_access() scans both the annotator declarations AND each intervention-point binding overlay to block system_prompt_file, api_key_env, aws_*_env, Rego bundle paths, and Cedar data paths. The overlay scan is critical — without it a clean declaration could be smuggled past by a binding that adds a system_prompt_file field.

  2. Dispatch-time: The url_sourced flag on LlmAnnotator and DefaultAnnotatorDispatcher blocks the provider-default credential env var fallback (e.g. OPENAI_API_KEY) which no field scan can see. This is the right second line since provider SDK credential discovery is transparent to a field-based scan.

Both layered correctly. A URL-sourced manifest cannot reach host filesystem or host credentials through any path.

edu-k12.yaml overlap: PRs #3115 and #3127 also fix these same three regex patterns. Whichever lands first, the other two will need the edu-k12 hunk removed on rebase. Not a blocker but worth coordinating.

LGTM on the ACS implementation.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: ACS remote/pinned system prompt and rego bundle

Approved

The core design is sound:

Security guards -- the url_sourced flag that blocks host-credential fallback for URL-sourced manifests is the right call. A remote manifest controlling both the LLM endpoint and the prompt URL could otherwise exfiltrate an API key or AWS session token; the flag prevents that path cleanly. The fail-closed behavior on fetch/read errors is correct.

Trust gate reuse -- fetch_pinned_https_text is the right primitive to reach for (HTTPS-only, sha256-pinned, Limits-bound). Adding it to dispatchers instead of reinventing the fetch path keeps the attack surface bounded.

Tests -- url_sourced_bedrock_without_inline_credentials_fails_closed and missing_system_prompt_file_fails_closed cover the two most important error paths. Good.

Note: edu-k12.yaml overlap with PR #3127

This PR also fixes three regex patterns in examples/policy-templates/edu-k12.yaml. PR #3127 fixes the same three rules (and additionally fixes the quorum vote recording bug). Both fix the failing tests, but the patterns differ slightly:

  • asi03: this PR uses (to\s+|an?\s+)?, #3127 uses (to\s+)?(a\s+|an\s+)? (equivalent)
  • cipa: this PR ends the second branch at kill|make|build|create|assemble, #3127 extends it to the full (make|build|create|assemble)\s+(a\s+)?(bomb|weapon|explosive|gun|knife) pattern (more specific)
  • asi09: this PR adds information to the keyword list, #3127 drops \s+ to .* (both fix the gap)

Whichever merges second will need to rebase. The ACS feature code itself is clean; this is only a coordination note.

…tcher factories

Re-review of the maintainer findings showed the URL-manifest credential-exfil
fix only reached the C ABI FFI path. The Rust, Python, and Node SDKs each expose
Manifest::from_url (which sets url_sourced = true) but then built the annotator
dispatcher via default_annotator_dispatcher(), which hardcoded url_sourced =
false, so the High-severity host-credential suppression was silently bypassed on
three of the four host surfaces.

- Make default_annotator_dispatcher_for(manifest, limits) the single factory and
  repoint the Rust, Python, and Node SDK host paths at it so provenance flows
  from the manifest on every surface.
- Delete the provenance-free factories and constructors that hardcoded
  url_sourced = false (default_annotator_dispatcher, *_with_limits, and
  DefaultAnnotatorDispatcher::new / ::with_limits). The type can no longer be
  constructed without a manifest, so provenance cannot be dropped again. This
  also removes the dead factory variants left over from the prior round.
- Inline the dead default_policy_dispatcher_with_limits into
  default_policy_dispatcher.
- Add regression tests: from_url_marks_manifest_url_sourced (load sets the flag,
  string load does not) and dispatcher_stores_url_sourced_provenance.

The surviving limits parameter is still Limits::default() at every call site
because no FFI or SDK builder limits knob exists yet; that host knob remains the
tracked Finding 4 follow-up.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
@MohammadHaroonAbuomar

Copy link
Copy Markdown
Collaborator

The dispatch-time flag is only wired into the C-FFI entry point (ffi.rs:514 → default_annotator_dispatcher_for(manifest, …), hence .NET is covered). The Rust, Python, and Node native bindings still call the plain default_annotator_dispatcher() (host/mod.rs:114, python/src/lib.rs:313, node/native/lib.rs:288), which hard-codes url_sourced=false. All three SDKs expose from_url. So, a host that loads a manifest via the Rust/Python/Node from_url and uses the built-in annotator dispatcher will still let a remote manifest's llm annotator read a provider-default credential and ship it to its attacker-chosen endpoint. (Explicit *_env fields are still blocked there by the load-time check — only the default fallback leaks.)

You can defer the other comments.

liamcrumm and others added 3 commits June 22, 2026 19:56
…t-env exfil)

Deep review found the URL-manifest credential-exfil fix closed only the llm
dispatcher sink. A URL-sourced (untrusted) manifest can still declare a rego
bundle_url; the bundled OPA dispatcher fetches the attacker-chosen pinned bundle
and runs it via opa eval, which inherits the host environment (opa.runtime().env)
and permits arbitrary egress (http.send). Reproduced: an attacker bundle leaked
AWS_SECRET_ACCESS_KEY to a local sink. This is the same exfil class as the llm
finding but broader (any env var, arbitrary network).

- Reject a rego bundle_url on a URL-sourced manifest at load
  (PolicyConfig::reject_url_sourced_remote_bundle, wired into
  reject_url_sourced_local_access over the fully extends-merged manifest). The
  hash pin does not establish trust because the same untrusted manifest chooses
  both the URL and the pin. bundle_url stays fully available to file-sourced,
  operator-authored manifests.
- Add regression test from_url_rejects_remote_rego_bundle_url (URL-sourced
  bundle_url rejected at load; file-sourced bundle_url still valid).
- Spec 2.3: document the bundle_url prohibition and its rationale; remove
  bundle_url from the list of forms a URL-sourced manifest may use.
- Spec 2.2: soften the SSRF wording from 'a fetch cannot target' to 'the
  validated fetch URL cannot name', and state that hostname resolution, DNS
  rebinding, and redirect hops are not revalidated (honest residual scope).
- Fix rustdoc on the default dispatcher factories that overclaimed 'host
  effective Limits' when callers pass Limits::default() (no host knob wired yet).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
Close the two partial items from @MohammadHaroonAbuomar's review.

Finding 2 (SSRF redirects): the initial fetch URL was IP-checked but ureq
followed redirects with only https_only, so a vetted public URL could 302 to a
loopback, link-local, or internal HTTPS host the initial guard would reject.
HttpExtendsFetcher now sets redirects(0) and follows redirects itself, re-running
validate_url_components (HTTPS plus the SSRF IP block) on every hop before
following it, with the hop count capped. This covers all four URL fetch paths
(from_url, extends, system_prompt_url, bundle_url) since they share the fetcher.

Finding 4 (host limits): every host call site passed Limits::default(), so a
host that tightened max_manifest_url_bytes / manifest_url_timeout_ms /
max_manifest_url_redirects had them ignored at dispatch-time fetches. Add a
focused URL-fetch-limits knob on all four host surfaces:
- FFI: acs_builder_set_url_fetch_limits + AcsBuilder.limits, threaded to both
  default dispatcher factories.
- Rust SDK: from_url_with_limits / from_manifest_with_dispatchers_and_limits.
- Python: optional max_url_bytes / url_timeout_ms / max_url_redirects on from_url.
- Node: optional urlFetchLimits on AgentControl.fromUrl.
Re-add default_policy_dispatcher_with_limits (now with real callers).

Tests: redirect-hop re-validation (loopback-https and http-downgrade Location
both blocked) and cap; FFI limits setter + build; Rust/Python/Node from_url
limits threading. Update spec 2.2 to state redirects are re-validated.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>
cspell flagged 'exfiltrates' on a changed line; reword to 'sends it out'
without changing meaning.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Liam Crumm <liamcrumm@gmail.com>

@MohammadHaroonAbuomar MohammadHaroonAbuomar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the comments

@MohammadHaroonAbuomar
MohammadHaroonAbuomar merged commit 8519313 into main Jun 22, 2026
144 checks passed
@MohammadHaroonAbuomar
MohammadHaroonAbuomar deleted the liamcrumm/acs-remote-prompt-and-bundle branch June 22, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation size/XL Extra large PR (500+ lines) tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants