Engine API: FastAPI reference adapter + /policies gap-fix (Epic 0, issue 3/32)

[Ricky-G]

Tracker: #2729
Read first: docs/studio/engine-api-contract.md §7 (Endpoint Catalog), §10 (Errors), §11 (Pagination); agentmesh.engine_api public surface from #3026 (PR #3027).

Summary

Stand up the reference FastAPI adapter for the Engine API contract that landed in PR #3013. One app, one process, all the v1 HTTP routes from §7. Every route uses the capability_flags decorator from agentmesh.engine_api and the generated OpenAPI document carries the x-capability-flags extension on every operation. This issue also fixes the long-standing counts-only gap on GET /api/v1/policies in agent-governance-python/agent-mesh/src/agentmesh/server/policy_server.py (lines 168 ff.).

This is the largest single issue in Epic 0. Treat it as the canonical reference implementation that all downstream Studio panels (Epics 2-9) will be wired against.

Deliverables

Path	Purpose
agent-governance-python/agent-mesh/src/agentmesh/engine_api/app.py	FastAPI app factory create_app() returning a fully wired FastAPI instance with inject_capability_extension already applied to its OpenAPI generator.
agent-governance-python/agent-mesh/src/agentmesh/engine_api/routes/	One module per route group: health.py, policies.py, policy_ops.py, audit.py, trust.py, agents.py, decisions.py, versions.py.
agent-governance-python/agent-mesh/src/agentmesh/engine_api/errors.py	Error envelope model + handler that maps HTTPException and validation errors to §10.2 shape with §10.3 codes.
agent-governance-python/agent-mesh/src/agentmesh/engine_api/pagination.py	PaginationParams dependency + Pagination response model per §11.
agent-governance-python/agent-mesh/src/agentmesh/engine_api/models.py	Pydantic response models: PolicySummary, PolicyDetail, AuditEntry, TrustScore, TrustGraph, AgentSummary, DecisionEntry, VersionsResponse, etc. Match field names from §7 exactly.
agent-governance-python/agent-mesh/tests/engine_api/test_app.py	App factory + capability extension wiring.
agent-governance-python/agent-mesh/tests/engine_api/test_routes_*.py	One test module per route group. Use fastapi.testclient.TestClient.
agent-governance-python/agent-mesh/tests/engine_api/test_errors.py	Envelope shape, all standard codes from §10.3.
agent-governance-python/agent-mesh/tests/engine_api/test_pagination.py	Query parsing + response object shape from §11.
agent-governance-python/agent-mesh/tests/engine_api/test_openapi_routes.py	Generated OpenAPI doc has x-capability-flags on every operation; allowlist derivation produces the spec-mandated 11-route list (per §6.2).

Scope (in)

1. Implement all 11 read-only routes from §7.1-7.3, §7.4-7.5, §7.7-7.12 plus the single mutating route POST /api/v1/policy/save (§7.6) — 12 HTTP operations total. Every route decorated with @capability_flags(...) using the exact flag values from §5.2.
2. Fix GET /api/v1/policies to return paginated PolicySummary objects per §7.2, not the current totals dict (policy_server.py:168).
3. Wire inject_capability_extension so GET /openapi.json carries x-capability-flags on every operation.
4. Error envelope (§10.2). errors.py defines the full §10.3 code enum (POLICY_NOT_FOUND, POLICY_PARSE_ERROR, FIXTURE_LOAD_ERROR, VALIDATION_ERROR, UNAUTHORIZED, RATE_LIMITED, ENGINE_UNAVAILABLE, INTERNAL_ERROR). This issue's routes can actually emit four of them: POLICY_NOT_FOUND, POLICY_PARSE_ERROR, FIXTURE_LOAD_ERROR, VALIDATION_ERROR. UNAUTHORIZED is reserved for the auth layer (issue build(deps): Bump hono from 4.11.7 to 4.12.5 in /packages/agent-os/extensions/mcp-server #7) — define the code, don't wire it here. FastAPI's default RequestValidationError handler must be overridden to emit the envelope shape.
5. Pagination (§11) on the 5 endpoints §11.3 marks paginated: /policies, /audit/log, /trust/scores, /agents, /decisions. PaginationParams as a FastAPI dependency (page default 1 min 1; limit default 20 min 1 max 100). Response wrapper is top-level items + pagination object (page, limit, total, has_next) per §11.2.
6. Backends for routes that don't have one yet return spec-conformant empties — do not invent fake data, mark each with a single # TODO(#<followup-when-filed>): comment pointing at the owning epic:
   • Paginated empties (items: [], pagination with total: 0): /audit/log (Epic 9), /trust/scores (Epic 8), /agents (Epic 8), /decisions (Epic 7).
   • /trust/graph (§7.9) is not paginated — it returns a graph object { "nodes": [], "edges": [] } (Epic 8).
7. GET /api/v1/versions (§7.12) returns: api: "1.0.0", engine: <agent-mesh package version via importlib.metadata>, python: <optional runtime version>, capabilities: <optional array of capability identifier strings>. Wire api and engine against live values, not hardcoded literals.
8. Sidecar entry point: python -m agentmesh.engine_api that runs uvicorn against create_app(), default bind 127.0.0.1:8080 (the spec §4 default). Port configurable via --port. Loopback-only by default (host arg defaults to 127.0.0.1, not 0.0.0.0).

Scope (out)

• WebSocket /api/v1/events (§12) — that's Epic 7a, issue build(deps-dev): Bump eslint from 8.57.1 to 10.0.2 in /packages/agent-os/extensions/copilot #16. Do not implement, do not register, do not stub.
• POST /api/v1/policy/reload (§8.1) — explicitly forbidden by spec. Do not add.
• Authentication / token handling — loopback-only is intentional for v1. Token enforcement for non-loopback bind lands in Epic 1b (issue build(deps): Bump hono from 4.11.7 to 4.12.5 in /packages/agent-os/extensions/mcp-server #7, agt serve).
• Deletion or modification of the five existing sidecar apps (policy_server.py, sidecar.py, audit_collector.py, trust_engine.py, api_gateway.py). They stay in tree for backward compatibility. Their deprecation is Epic 10 (issues fix: address Dependabot and CodeQL security alerts #23-chore: add CODEOWNERS and SBOM generation for CELA compliance #27).
• AGT CLI integration (agt serve) — Epic 1b, issue build(deps): Bump hono from 4.11.7 to 4.12.5 in /packages/agent-os/extensions/mcp-server #7.
• Real implementations for trust/audit/agents/decisions backends — those are their own epics. Item 6 above defines the placeholder contract.
• Streamlit dashboards, HTML pages, any UI work. This issue produces an HTTP surface only.

Acceptance criteria

• python -c "from agentmesh.engine_api import create_app; create_app()" succeeds with no warnings.
• All 12 HTTP operations (11 read-only + POST /api/v1/policy/save) registered on the app. Verified by iterating app.routes in a test.
• Every registered route has __capability_flags__ set and the flag values match §5.2 of the contract doc exactly.
• app.openapi() returns a document where every operation under paths.*.[get|post|...] carries an x-capability-flags object with all three boolean fields.
• derive_studio_client_allowlist(app.openapi()) returns exactly the 11 routes listed in §6.2 of the contract, in sorted order. No more, no fewer.
• POST /api/v1/policy/reload does NOT appear in the app. A test asserts its absence.
• GET /api/v1/policies returns a paginated PolicySummary list when there are loaded policies and an empty paginated response otherwise. Test covers both.
• GET /api/v1/policies/{id} returns 404 with code: "POLICY_NOT_FOUND" envelope when id is unknown.
• POST /api/v1/policy/validate with malformed YAML returns 422 with code: "POLICY_PARSE_ERROR" envelope.
• GET /api/v1/versions returns api == "1.0.0" and a non-empty engine string. (capabilities is optional; assert it is a list if present.)
• GET /api/v1/trust/graph returns a { "nodes": [...], "edges": [...] } object, not a paginated items/pagination wrapper.
• All FastAPI RequestValidationError 422 responses are remapped to the envelope shape with code: "VALIDATION_ERROR". Test covers one validation failure end-to-end (assert the body is the envelope, not FastAPI's default {"detail": [...]}).
• Pagination parses per §11.1 (page default 1 / min 1; limit default 20 / min 1 / max 100). Out-of-range values produce a VALIDATION_ERROR envelope. The 4 placeholder list routes and /policies all return the items + pagination wrapper from §11.2.
• python -m agentmesh.engine_api --help prints usage; default bind is 127.0.0.1:8080. A test asserts the default host and port in the CLI parser.
• ruff check --select E,F,W --ignore E501 passes on changed files.
• pytest agent-governance-python/agent-mesh/tests/engine_api/ passes. All new test modules ship green.

Design notes

• No business-logic refactor in this PR. Where data is needed, import from existing modules (e.g., the policy registry helpers in policy_server.py, agentmesh.governance.audit, agentmesh.services.registry.agent_registry). Do not move or rewrite them. If a clean source isn't available, use the empty placeholder pattern from scope item 6.
• One app, one OpenAPI doc. Resist the temptation to mount sub-apps. The whole point of this issue is to give Studio one URL to discover the entire surface.
• capability_flags is the ONLY way flags get attached. Do not hand-set __capability_flags__. Do not write your own decorator. The library from Engine API: capability metadata implementation (Epic 0, issue 2/32) #3026 is the contract.
• POST /api/v1/policy/save is the only operation in this app that mutates. Make this loud in code: keep it in its own route module (policy_ops.py) and add a module-level comment naming the spec invariant. Per §8.1, save triggers a policy reload as a side effect — that is the intended reload path, and the reason the standalone POST /api/v1/policy/reload route is excluded. Wire save so it persists then reloads; do not expose reload separately.
• Error envelope handler ordering matters. Register the validation handler before the generic handler, or FastAPI's default will win. Add an integration test that confirms a RequestValidationError produces the envelope, not FastAPI's default {"detail": [...]}.
• Pydantic version: PR feat(engine-api): add capability-metadata library for AGT Studio #3027 settled this. Use the same import pattern (pydantic.BaseModel) and the same version pin already in agent-mesh/pyproject.toml. Don't bump it.
• Tests must use TestClient, not live uvicorn. Spawning a real server in CI is flaky on Windows runners.

References

• Contract spec: docs/studio/engine-api-contract.md §5, §6, §7, §10, §11
• OpenAPI doc: docs/studio/openapi.yaml
• Capability library: agentmesh.engine_api (PR feat(engine-api): add capability-metadata library for AGT Studio #3027, merged 2026-06-15)
• Counts-only gap: agent-governance-python/agent-mesh/src/agentmesh/server/policy_server.py lines 165-175

Notes for picking this up

• Run git fetch origin main && git log origin/main --oneline -5 before starting. Confirm PR feat(engine-api): add capability-metadata library for AGT Studio #3027 is in.
• Implement the route groups in this order so each PR-internal commit is reviewable: errors.py -> pagination.py -> models.py -> health.py -> versions.py -> policies.py -> policy_ops.py -> audit.py -> trust.py -> agents.py -> decisions.py -> app.py -> CLI entry point.
• app.py exposes create_app(). Assemble the FastAPI instance, register every route module, then apply inject_capability_extension to the OpenAPI generator last so the extension sees all operations.
• PR title MUST end with (Epic 0, issue #<this-issue-number>) so GitHub auto-closes on merge.
• PR description must follow the repo template in .github/AGENTS.md (Summary / Problem / Changes table / Testing).
• Expected PR size: ~1200-1800 LOC including tests. If you find yourself over 2500, stop and split.

[github-actions]

🟡 Contributor Check: MEDIUM

Check	Result
Profile	MEDIUM
Credential	LOW
Overall	MEDIUM

Automated check by AGT Contributor Check.

[DhineshPonnarasan]

Hi Ricky Gummadi (@Ricky-G)

I spent some time reviewing #3066 and the related work before starting implementation.

What I found

• This issue depends on the Engine API contract from PR docs: engine API contract spec + OpenAPI 3.1 document (Epic 0, issue #3011) #3013 and the capability metadata library from PR feat(engine-api): add capability-metadata library for AGT Studio #3027.
• The capability metadata implementation already provides the pieces needed for capability flags, OpenAPI extensions, and Studio allowlist generation, so I plan to reuse those directly.
• The current /api/v1/policies endpoint only returns summary counts and does not match the PolicySummary response defined in the contract.
• Pagination requirements are clearly defined in the contract and apply to the list endpoints called out in the issue.
• FastAPI's default validation errors will need to be wrapped in the Engine API error envelope format required by the spec.

Planned approach

• Build the FastAPI reference adapter using the existing capability metadata library.
• Implement the route groups and response models based on the Engine API contract.
• Update the /policies endpoint to return contract-compliant PolicySummary results instead of counts-only data.
• Return spec-compliant empty responses for the audit, trust, agents, and decisions endpoints until their respective epics provide real backends.
• Add tests to verify capability metadata, OpenAPI generation, pagination, error handling, and Studio allowlist behavior.

One question

While reviewing the contract, I noticed that /api/v1/events is reserved in the spec, but this issue also says "do not implement, do not register, do not stub" that endpoint. My plan is to follow the issue scope as written, but I wanted to mention the difference in case there is any additional guidance.

If this approach looks good, shall I go ahead and work on it and open a PR?

[Ricky-G]

Hi Dhinesh Ponnarasan (@DhineshPonnarasan), thanks for the careful read before diving in. Your plan lines up with the contract, so please go ahead.

Reusing the capability metadata pieces from #3027 is the right call, and wrapping FastAPI's default validation errors in the Engine API error envelope is the part most likely to bite, so good that you already have it on the radar.

One scope guardrail: please keep /api/v1/events out of this issue. Epic 0 issue 3/32 is the FastAPI app factory, the v1 route surface from the endpoint catalog, and the counts-only GET /api/v1/policies PolicySummary gap-fix. The events stream has its own design constraints and is tracked separately, so this PR should not stand up an /api/v1/events implementation, not even a placeholder. If the catalog wiring tempts you to stub it, please leave it as a documented gap instead.

[DhineshPonnarasan]

Hi Ricky Gummadi (@Ricky-G),

I did one final pass against the issue, contract, OpenAPI spec, and your earlier guidance.

I initially considered reducing the route surface to only the routes backed by existing implementations, but after re-reading the issue and endpoint catalog I agree the correct scope is the full v1 route surface (excluding /api/v1/events), with the audit/trust/agents/decisions endpoints returning the spec-defined placeholder responses until their owning epics provide real backends.

The current implementation plan is therefore:

• Full endpoint catalog from §7 (12 operations total).
• /api/v1/events remains unregistered.
• /policies updated to return contract-compliant PolicySummary responses.
• Placeholder responses for audit/trust/agents/decisions as described in the issue.
• Capability metadata and OpenAPI extensions wired through the existing engine_api capability library.
• Contract-compliant error envelopes and pagination behavior.

I'll proceed with the implementation following that scope unless you see anything I missed.

[Ricky-G]

Hi Dhinesh Ponnarasan (@DhineshPonnarasan) apologies hopefully you have not started work on this, I have miss timed the agent, the PR has already been done but this is the 3rd ticket from this epic.

#2729

If you are keen I will create the 4th ticket from here and you can take this on and do the PR for that, sorry for this one the pr is done I should have checked befor hand.

[DhineshPonnarasan]

Hi Ricky Gummadi (@Ricky-G),

No worries at all, and thanks for letting me know. I had only completed the investigation and design review so far, so no code changes were made.

I'd definitely be interested in taking on the next ticket in the epic when it's available. Please feel free to assign it to me once it's created, and I'll take a look.

Thanks again for the heads-up!