Skip to content

schemas: publish external target protocol schemas - #709

Merged
flyingrobots merged 2 commits into
mainfrom
schemas/external-target-protocol
Jul 5, 2026
Merged

flyingrobots merged 2 commits into
mainfrom
schemas/external-target-protocol

Conversation

@flyingrobots

Copy link
Copy Markdown
Owner

Branch / Issue-Title Check

Branch schemas/external-target-protocol and title schemas: publish external target protocol schemas match #708 and use no reserved prefixes.

Why

#680 now has an external-process protocol spec and PR #707 shipped descriptor verification, but target authors still lacked canonical JSON Schemas for the protocol envelopes.

Changes

  • Add JSON Schemas for target descriptors, requests, responses, diagnostics, and artifact manifests.
  • Validate representative protocol artifacts through generated_json_artifacts.
  • Add negative schema coverage for drive-like paths and dot-only target/artifact paths.
  • Update the schema index, protocol reference, and changelog.

Closes #708
Related to #680

Risk

Low. This publishes schema artifacts and tests only. It does not add target run, execute target code, sandbox processes, or copy artifacts.

Backout

Revert 03534e38 to remove the schema files, protocol schema tests, and documentation references.

Testing

  • cargo test -p wesley-core --test generated_json_artifacts external_target_protocol_artifacts_satisfy_declared_schemas (RED before schemas, GREEN after schemas)
  • cargo test -p wesley-core --test generated_json_artifacts
  • cargo fmt --check
  • pnpm exec prettier --check schemas/README.md docs/reference/external-target-protocol.md CHANGELOG.md schemas/wesley-target-descriptor-v1.schema.json schemas/wesley-target-request-v1.schema.json schemas/wesley-target-response-v1.schema.json schemas/wesley-target-diagnostic-v1.schema.json schemas/wesley-target-artifact-manifest-v1.schema.json
  • cargo xtask docs-check
  • git diff --check
  • pnpm run preflight
  • pre-push Rust product preflight

@coderabbitai

coderabbitai Bot commented Jul 5, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

@flyingrobots, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 15 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9f0bcefd-e245-4a5d-92b7-31e94431e0b3

📥 Commits

Reviewing files that changed from the base of the PR and between a9acea3 and b89d4a0.

📒 Files selected for processing (9)
  • CHANGELOG.md
  • crates/wesley-core/tests/generated_json_artifacts.rs
  • docs/reference/external-target-protocol.md
  • schemas/README.md
  • schemas/wesley-target-artifact-manifest-v1.schema.json
  • schemas/wesley-target-descriptor-v1.schema.json
  • schemas/wesley-target-diagnostic-v1.schema.json
  • schemas/wesley-target-request-v1.schema.json
  • schemas/wesley-target-response-v1.schema.json
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch schemas/external-target-protocol

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 5, 2026 •

Copy link
Copy Markdown

🔍 The Case of Pull Request #709

Schema Sets

  • ecommerce
  • reference

Schema Set ecommerce

Plain-English Readout

  • Holmes (evidence investigation): The Holmes report is unavailable because the workflow finished without a readable holmes-report.json artifact.
  • Watson (independent verification): The Watson report is unavailable because the workflow finished without a readable watson-report.json artifact.
  • Moriarty (trend forecast): The Moriarty forecast is unavailable because the workflow finished without a readable moriarty-report.json artifact.

Suggested next actions

  1. Regenerate the HOLMES artifacts and make sure holmes-report.json is uploaded before trusting this PR summary.
  2. Regenerate the WATSON artifacts and make sure watson-report.json is uploaded before trusting this PR summary.
  3. Regenerate the MORIARTY artifacts and make sure moriarty-report.json is uploaded before trusting this PR summary.
🕵️ SHA-lock HOLMES full report for ecommerce (click to expand)

Report unavailable for holmes: readable holmes-report.md artifact not found.

🩺 Dr. WATSON full report for ecommerce (click to expand)

Report unavailable for watson: readable watson-report.md artifact not found.

🔮 Professor MORIARTY full report for ecommerce (click to expand)

Report unavailable for moriarty: readable moriarty-report.md artifact not found.


Schema Set reference

Plain-English Readout

  • Holmes (evidence investigation): The Holmes report is unavailable because the workflow finished without a readable holmes-report.json artifact.
  • Watson (independent verification): The Watson report is unavailable because the workflow finished without a readable watson-report.json artifact.
  • Moriarty (trend forecast): The Moriarty forecast is unavailable because the workflow finished without a readable moriarty-report.json artifact.

Suggested next actions

  1. Regenerate the HOLMES artifacts and make sure holmes-report.json is uploaded before trusting this PR summary.
  2. Regenerate the WATSON artifacts and make sure watson-report.json is uploaded before trusting this PR summary.
  3. Regenerate the MORIARTY artifacts and make sure moriarty-report.json is uploaded before trusting this PR summary.
🕵️ SHA-lock HOLMES full report for reference (click to expand)

Report unavailable for holmes: readable holmes-report.md artifact not found.

🩺 Dr. WATSON full report for reference (click to expand)

Report unavailable for watson: readable watson-report.md artifact not found.

🔮 Professor MORIARTY full report for reference (click to expand)

Report unavailable for moriarty: readable moriarty-report.md artifact not found.


📚 Glossary (what the Holmes terms mean)
  • HOLMES: Wesley’s main evidence investigation. It decides whether the cited proof is strong enough to justify shipping this commit.
  • WATSON: An independent verification pass. It checks Holmes’s citations and score math instead of trusting them blindly.
  • MORIARTY: A readiness forecast over time. It is advisory trend analysis, not the release gate itself.
  • Schema coverage score (SCS): How much of the schema has direct supporting evidence across generated artifacts and cited proof.
  • Test confidence index (TCI): How much test evidence exists for constraints, policies, relationships, and operations.
  • Migration risk index (MRI): How risky the schema change is to roll out. Lower is better.
  • Evidence trust: Whether the report is backed by exact citations, whole-file citations, or coarse references. Weak trust means the claim may be directionally right but not specific enough to trust blindly.
  • Citation quality: A count of exact line-span citations versus whole-file or coarse references.
  • ELEMENTARY: Ready to ship based on the current evidence.
  • REQUIRES INVESTIGATION: More work or review is needed before shipping.
  • YOU SHALL NOT PASS: Do not ship this change in its current state.

Machine-readable reports are grouped by schema set in workflow artifacts.


Filed at 221B Repository Street

@flyingrobots

Copy link
Copy Markdown
Owner Author

@codex review please

Self-Code Review Findings

ID Severity File Lines Issue Mitigation
SELF-709-1 P3 schemas/wesley-target-artifact-manifest-v1.schema.json, docs/reference/external-target-protocol.md schema items.uniqueItems; docs artifact rules The artifact manifest rules say repeated paths are invalid, but draft-07 JSON Schema uniqueItems only rejects exact duplicate artifact objects. A manifest with the same path and different kind/sha256 still requires host/runtime semantic validation. Add an explicit schema-boundary note and regression coverage for exact duplicate object rejection plus the intentionally schema-external duplicate-path semantic rule, so future target run work cannot mistake shape validation for full artifact conflict validation.

@flyingrobots

Copy link
Copy Markdown
Owner Author

Activity Summary

Item Severity Source File(s) Commit Outcome
#708 P1 Issue schemas/wesley-target-*.schema.json, crates/wesley-core/tests/generated_json_artifacts.rs, docs/reference/external-target-protocol.md, schemas/README.md, CHANGELOG.md 03534e38 Published canonical external target protocol JSON Schemas, schema validation coverage, and docs links.
SELF-709-1 P3 Self-review schemas/wesley-target-artifact-manifest-v1.schema.json, docs/reference/external-target-protocol.md, crates/wesley-core/tests/generated_json_artifacts.rs b89d4a0a Documented that JSON Schema rejects exact duplicate artifact objects while host validation must reject repeated paths with different metadata; added regression coverage.

Verification:

  • RED: cargo test -p wesley-core --test generated_json_artifacts external_target_protocol_artifacts_satisfy_declared_schemas -- --nocapture failed before the docs boundary was added.
  • GREEN: cargo test -p wesley-core --test generated_json_artifacts passed.
  • GREEN: cargo fmt --check passed.
  • GREEN: pnpm exec prettier --check schemas/wesley-target-artifact-manifest-v1.schema.json docs/reference/external-target-protocol.md passed.
  • GREEN: git diff --check passed.
  • GREEN: pnpm run preflight passed.
  • GREEN: GitHub PR checks are passing.

Merge gate status: open except for the repository review policy (REVIEW_REQUIRED), which requires admin merge for this single-maintainer flow.

@flyingrobots
flyingrobots merged commit cd29f4f into main Jul 5, 2026
20 checks passed
@flyingrobots
flyingrobots deleted the schemas/external-target-protocol branch July 5, 2026 11:40

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 03534e3811

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

"apiVersion": { "const": "wesley.target-artifact-manifest/v1" },
"items": {
"type": "array",
"uniqueItems": true,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 Badge Reject duplicate artifact paths by path

For manifests where a target returns two items with the same path but a different kind or sha256, draft-07 uniqueItems compares the whole artifact object, so both entries still satisfy this schema even though the protocol says repeated paths are invalid and later copy-out validation must treat them as a hard failure. Since this file is published as the machine-readable artifact-manifest contract, consumers that gate on schema validation alone can accept conflicting outputs; either make the schema boundary explicit and add semantic-validation coverage, or enforce path-level uniqueness outside this array-level check.

Useful? React with 👍 / 👎.

},
"capabilities": {
"type": "object",
"required": ["inputs", "outputs", "requiresNetwork", "requiresAmbientFilesystem"],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Align schema with target verify defaults

When a descriptor omits requiresNetwork and requiresAmbientFilesystem, the current target verify implementation treats both fields as false via serde defaults and accepts the descriptor, but this required list makes the published schema reject the same input before it reaches the CLI. That leaves target authors with two conflicting validators for the same wesley.target-descriptor/v1 contract; either require the fields in the CLI as well or relax the schema to match the shipped verifier defaults.

Useful? React with 👍 / 👎.

},
"workspaceRelativePath": {
"type": "string",
"pattern": "^(?!/)(?!.*(^|/)\\.\\.(/|$))(?!.*[:\\\\\\u0000])(?=.*[^./]).+"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Anchor path regexes across line breaks

For artifact paths containing a newline before a forbidden suffix, e.g. generated/hello\n/../escape, ECMA-style . in JSON Schema patterns does not cross the newline and this pattern has no end anchor, so it can match only the safe prefix and still validate the full string. That bypasses the documented ban on parent-directory segments, backslashes, colons, and NULs for consumers relying on the published schema; anchor the pattern to the whole value and avoid .* assumptions across line terminators or reject line terminators explicitly.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

schemas: publish external target protocol schemas

1 participant