Repository navigation
Bind progressive MCP discovery to versioned catalog checkpoints #566
Description
Activity
Hi,
I’ve been looking at #566 from a conformance angle and did a read-only pass over the current call/catalog path. There are already useful partial controls for catalog membership, advertised-vs-approved schema identity, and catalog drift, but the integrated #566 discovery/checkpoint/call-basis path is not built yet.For that future path, one fixture shape may be useful: keep three identities independently mutable — the discovered schema, the approved catalog checkpoint, and the checkpoint attributed to a governed call.
That gives three focused differentials: unapproved late discovery, schema-only substitution under the same apparent tool identity, and a historical call being rebound to a later checkpoint.
The aim is not to change the restart/immutability model, only to make those identities independently testable as #566 lands. Would that fixture shape add useful coverage to the acceptance matrix?
The discovered schema, approved checkpoint, and call-attributed checkpoint need independent acceptance cases. Keep the suggested conformance matrix here; implementation ownership is not established by a review comment alone. Coordinate the manifest side with agentrust-io/agent-manifest#340.
Building on the three independently mutable identities already retained here, I think the acceptance matrix can add two things without repeating the existing fixture proposal:
-
Make the discovery→checkpoint pipeline explicit. Separate acquisition, structural validation, projection, and comparison. MCP
tools/listis paginated, while current cMCP5b63642consumes one returnedtoolsarray in this path; discovery also collapses tools by name, and the portable checkpoint byte recipe remains a design input. Could the matrix require pagination exhaustion with repeated/cyclic-cursor controls, refusal of duplicate or malformed discovery input before projection, and one pinned canonical byte recipe before comparison? -
Add a legitimate-advance positive control. Alongside the existing late-discovery, schema-substitution, and historical-rebinding negatives, include C0→C1 where a historical call remains correctly attributed to approved C0 and a later call uses separately approved C1. That distinguishes correct evolution from an implementation that passes by rejecting every change. It does not imply C0 remains authorized for new calls, or that historical transition provenance alone establishes current authorization.
Manifest-side digest identity, root↔leaf consistency, transition-evidence join, and the portable vector live on agentrust-io/agent-manifest#340. This note only extends the #566 acceptance shape; it does not restate the three-identity proposal, change the restart/immutability model, or claim implementation ownership.
-
Both in. The second one is the more important of the two and I want to say why before the first.
The legitimate-advance positive control is adopted
C0 to C1, with the historical call still attributed to approved C0 and the later call using separately approved C1. Your reason is the right one: without it the matrix cannot tell correct evolution from an implementation that passes by rejecting every change. A set composed entirely of negatives is satisfied by a verifier that refuses everything, and that verifier scores identically to a correct one.
This is the third instance of the same gap in this org in a week, which is why it is worth naming as a rule rather than a fixture:
agentrust-io/trace-spec#306's vector set could only expect acceptance,agentrust-io/cmcp#630covers six execution-binding vectors that a code-point sorter passes, and now this. Every refusal in an acceptance matrix needs a conformant twin, and the twin has to force the condition the refusal is testing.Your two caveats are kept as written, because they are what stops the control being over-read: it does not imply C0 remains authorized for new calls, and historical transition provenance alone does not establish current authorization.
The four-stage pipeline is adopted
Acquisition, structural validation, projection, comparison, as separate stages with the matrix requiring:
- pagination exhaustion, with repeated-cursor and cyclic-cursor controls;
- refusal of duplicate or malformed discovery input before projection;
- one pinned canonical byte recipe before comparison.
Separating them matters because a failure in acquisition and a failure in comparison mean different things and a single "discovery failed" result supports neither. Same discipline as the per-boundary failure reporting ruled on #568.
Your pagination observation is correct, and it is worse than a matrix gap
Verified on
main:_upstream_toolsbuildstools/listwith an empty params object, readsresult.get("tools")once, and returns it. The stringcursorappears zero times in the entiresrc/tree, so there is no pagination handling anywhere in the codebase rather than only in this path.So a server that returns a first page plus
nextCursorgives cMCP a partial catalog today, and_check_upstream_driftthen compares an approved catalog against a truncated advertisement. That is a live defect rather than a test-design question, and a tool omitted from page two is invisible in exactly the direction that matters. Filing it separately so it is not gated on the matrix landing.Thanks, and sorry for the late response. My bad.
Following the coordinated projection proposal on agentrust-io/agent-manifest#340, I suggest retaining five bounded acceptance obligations here.
At the reviewed cMCP pin e019431, #633's pagination controls are present, and close/reset cleanup invalidates the discovery, provenance and drift-check caches. These are existing regression surfaces, not proof of the broader progressive-discovery/checkpoint contract. Cache invalidation alone does not establish successful completion of cleanup or the session transition.
This is a proposed acceptance design, not newly executed fixture results. AM-facing cases remain conditional on adoption of that projection profile.
1. Projected content
A valid approved-runtime/AM pair matches after independently checking both commitments. Schema and description negatives change the proposed canonical projection preimage, hold AM A and its leaves fixed, and honestly recompute runtime
definition_hashand D. They must fail comparison — not merely fail because D is stale.2. Complete population and join
Require authenticated, unique counterparts across both complete populations under the proposed name-and-endpoint join; no subset success. Distinguish invalid/ambiguous identities from otherwise valid missing or wrong counterparts. Do not introduce global AM tool-name uniqueness.
3. Discovery versus approval
Preserve late-discovery, schema-substitution, tool-removal, refresh and stale-cache cases against an independently selected approved checkpoint. Each refresh fixture declares its lifecycle boundary and context; refresh neither grants approval nor selects C1.
Keep valid complete-empty and exhausted-pagination controls alongside incomplete/malformed acquisition, duplicate discovered names, cursor cycles and page-budget refusal. An empty non-terminal page is not an empty listing.
This is the cMCP discovery/approval relation, not the AM projection.
4. Validation versus projectability
Reject malformed or duplicate discovery input before projection, even when detected inside the acquisition routine; distinguish otherwise valid material outside the selected profile.
Under the proposed SPIFFE join, a valid non-empty null-SPIFFE catalog is out of profile — not unauthorized — and cannot be filtered into a match.
5. Legitimate evolution
Preserve the adopted C0 → separately-approved-C1 positive control: historical calls remain attributed to C0; later C1 calls require their own authority. Reject historical rebinding and reliance on C0 provenance alone.
Boundary
Keep
acquisition → structural validation → projection → comparison, with a conformant twin for every refusal. Matching does not itself establish approval, freshness or call authority.Does this capture the intended acceptance shape, while leaving the AM-profile decision, broader refresh contract, implementation ownership and wire representation open?
@solloek369-arch accepted as the acceptance shape for this issue. The profile the AM-facing cases were waiting on is now ruled in agentrust-io/agent-manifest#340, so they can be written against it. Obligation 3 needs no cross-repository decision at all, which makes it the natural first fixture set.
@imran-siddique One boundary question before the Obligation 3 late-discovery fixture, on current main
c67f3b4cbeb2f323d28053861cb4f0b8be3aadce._check_upstream_driftcompares only names already present in the approved catalog. With an approvedlookup_customer, an otherwise matching advertised listing that also containslate_toolproduces the same drift result as the listing without it: the approved definition matches,late_toolis not added todrifted, and there is nocatalog_driftaudit entry.A direct call to
late_tooldoes not reach that comparison._call_tool_implreturns at catalog lookup withcatalog_miss(Tool 'late_tool' not in attested catalog). The gatewaytools/listis built from catalog entries, solate_toolis absent there too. On the approved name, a substituted schema isdefinition_changedand absence of the approved tool iswithdrawn.The remaining question is the discovery-versus-selected-checkpoint relation:
Should an additional unapproved advertised name make that relation a mismatch, or should extra advertised names stay outside this comparison and remain a catalog-admission matter?
Production is unchanged. Carrying a mismatch through the existing drift result would, under fail-closed, also deny an already-approved call to that same server. Once you rule the boundary, I will bind the late-discovery fixture to that layer.
@solloek369-arch extra advertised names stay outside the drift comparison. An unapproved
late_toolis not a mismatch, so approved calls to that server keep working, and admission stays the gate:catalog_missalready refuses the call itself.It is still evidence. Record advertised names that are not in the selected checkpoint as observed and unadmitted, so the late-discovery fact can be checked later without becoming a denial. Bind the Obligation 3 fixture to that: the same drift result with and without
late_tool, pluslate_toolin the recorded unadmitted set.@solloek369-arch @imran-siddique I have read the Obligation 3 discussion and the
ruling that extra advertised names remain outside drift comparison but must be
recorded as observed and unadmitted. At current main 4acb813,
the inspected drift path preserves approved-call behavior and catalog lookup
denies unknown names; I did not find unadmitted-name evidence recording in that
path. This is source inspection only, not executed fixture evidence.I can help with independent review of your fixture set: verify the approved-call
positive twin, recorded unadmitted-name evidence against the selected checkpoint,
catalog_miss with no forwarding, and preservation of schema-substitution/tool-removal
outcomes. A weakening that drops only the evidence should also be caught.
Which subset and pinned revision would be useful to review, and which evidence
representation should it assert? I am offering review support; implementation
ownership remains with the coordinated work.@dinakarjs yes. The recording is in #723, held for one change (a bound on how many unadmitted names a single listing can write). Once that lands, review it against the ruling above: the same drift result with and without
late_tool,late_toolin the recorded set, and a listing over the bound writing exactly the cap plus the count.
Support progressive discovery and server/discover without weakening catalog approval. Each governed call must bind to the exact discovered schema and approved catalog checkpoint. Acceptance evidence: cache refresh, stale catalog, tool removal, schema substitution, and unapproved late-discovery tests.