Skip to content

cli: add target descriptor verification - #707

Merged
flyingrobots merged 2 commits into
mainfrom
cli/target-descriptor-verify
Jul 5, 2026
Merged

flyingrobots merged 2 commits into
mainfrom
cli/target-descriptor-verify

Conversation

@flyingrobots

@flyingrobots flyingrobots commented Jul 5, 2026 •

Copy link
Copy Markdown
Owner

Branch / Issue-Title Check

Branch cli/target-descriptor-verify and title cli: add target descriptor verification match #705 and use no reserved prefixes.

Why

#680 named wesley target verify in the external-process protocol docs, but the native CLI did not ship a descriptor verifier. Third-party target authors need a non-executing validation surface before any target runner exists.

Changes

  • Add wesley target verify <descriptor> [--json].
  • Validate descriptor API/protocol versions, path-safe names, descriptor-relative command paths, bounded timeouts, required capabilities, denied network/ambient filesystem requests, and safe output directories.
  • Reject Windows drive-like colon paths so descriptor validation stays deterministic across hosts.
  • Add CLI tests for valid descriptors, non-execution, deterministic unsafe-descriptor diagnostics, malformed descriptor parse errors, and drive-like path rejection.
  • Update CLI/protocol docs and the changelog.

Closes #705
Related to #680

Risk

Medium-low. This adds a validation-only CLI command. It does not execute target code, add target run, perform process sandboxing, or copy artifacts.

Backout

Revert ac2f95c0 and 9368f48c to remove the target command and reference docs.

Testing

  • cargo test -p wesley-cli --test cli target_verify_reports_descriptor_parse_errors (RED before fix, GREEN after fix)
  • cargo test -p wesley-cli --test cli target_verify_rejects_windows_drive_like_paths (RED before fix, GREEN after fix)
  • cargo test -p wesley-cli --test cli target_verify
  • cargo test -p wesley-cli --test cli
  • cargo fmt --check
  • pnpm exec prettier --check docs/reference/cli.md docs/reference/external-target-protocol.md CHANGELOG.md
  • cargo xtask docs-check
  • git diff --check
  • pnpm run preflight
  • pre-push Rust product preflight

@coderabbitai

coderabbitai Bot commented Jul 5, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

@flyingrobots, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 39 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 1a95ec46-09ee-4450-a173-bd9d11550a62

📥 Commits

Reviewing files that changed from the base of the PR and between c20f92f and 9368f48.

📒 Files selected for processing (5)
  • CHANGELOG.md
  • crates/wesley-cli/src/main.rs
  • crates/wesley-cli/tests/cli.rs
  • docs/reference/cli.md
  • docs/reference/external-target-protocol.md
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch cli/target-descriptor-verify

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 5, 2026 •

Copy link
Copy Markdown

🔍 The Case of Pull Request #707

Schema Sets

  • ecommerce
  • reference

Schema Set ecommerce

Plain-English Readout

  • Holmes (evidence investigation): The Holmes report is unavailable because the workflow finished without a readable holmes-report.json artifact.
  • Watson (independent verification): The Watson report is unavailable because the workflow finished without a readable watson-report.json artifact.
  • Moriarty (trend forecast): The Moriarty forecast is unavailable because the workflow finished without a readable moriarty-report.json artifact.

Suggested next actions

  1. Regenerate the HOLMES artifacts and make sure holmes-report.json is uploaded before trusting this PR summary.
  2. Regenerate the WATSON artifacts and make sure watson-report.json is uploaded before trusting this PR summary.
  3. Regenerate the MORIARTY artifacts and make sure moriarty-report.json is uploaded before trusting this PR summary.
🕵️ SHA-lock HOLMES full report for ecommerce (click to expand)

Report unavailable for holmes: readable holmes-report.md artifact not found.

🩺 Dr. WATSON full report for ecommerce (click to expand)

Report unavailable for watson: readable watson-report.md artifact not found.

🔮 Professor MORIARTY full report for ecommerce (click to expand)

Report unavailable for moriarty: readable moriarty-report.md artifact not found.


Schema Set reference

Plain-English Readout

  • Holmes (evidence investigation): The Holmes report is unavailable because the workflow finished without a readable holmes-report.json artifact.
  • Watson (independent verification): The Watson report is unavailable because the workflow finished without a readable watson-report.json artifact.
  • Moriarty (trend forecast): The Moriarty forecast is unavailable because the workflow finished without a readable moriarty-report.json artifact.

Suggested next actions

  1. Regenerate the HOLMES artifacts and make sure holmes-report.json is uploaded before trusting this PR summary.
  2. Regenerate the WATSON artifacts and make sure watson-report.json is uploaded before trusting this PR summary.
  3. Regenerate the MORIARTY artifacts and make sure moriarty-report.json is uploaded before trusting this PR summary.
🕵️ SHA-lock HOLMES full report for reference (click to expand)

Report unavailable for holmes: readable holmes-report.md artifact not found.

🩺 Dr. WATSON full report for reference (click to expand)

Report unavailable for watson: readable watson-report.md artifact not found.

🔮 Professor MORIARTY full report for reference (click to expand)

Report unavailable for moriarty: readable moriarty-report.md artifact not found.


📚 Glossary (what the Holmes terms mean)
  • HOLMES: Wesley’s main evidence investigation. It decides whether the cited proof is strong enough to justify shipping this commit.
  • WATSON: An independent verification pass. It checks Holmes’s citations and score math instead of trusting them blindly.
  • MORIARTY: A readiness forecast over time. It is advisory trend analysis, not the release gate itself.
  • Schema coverage score (SCS): How much of the schema has direct supporting evidence across generated artifacts and cited proof.
  • Test confidence index (TCI): How much test evidence exists for constraints, policies, relationships, and operations.
  • Migration risk index (MRI): How risky the schema change is to roll out. Lower is better.
  • Evidence trust: Whether the report is backed by exact citations, whole-file citations, or coarse references. Weak trust means the claim may be directionally right but not specific enough to trust blindly.
  • Citation quality: A count of exact line-span citations versus whole-file or coarse references.
  • ELEMENTARY: Ready to ship based on the current evidence.
  • REQUIRES INVESTIGATION: More work or review is needed before shipping.
  • YOU SHALL NOT PASS: Do not ship this change in its current state.

Machine-readable reports are grouped by schema set in workflow artifacts.


Filed at 221B Repository Street

Copy link
Copy Markdown
Owner Author

Self-Code Review Findings

@codex please sanity-check this finding before merge.

ID Severity Source File Lines Issue Mitigation
SELF-707-1 P2 Self crates/wesley-cli/src/main.rs 1479-1520 Target descriptor path safety is not fully cross-host deterministic: on Unix, C:/tool and C:/generated are parsed as relative paths and would pass despite being Windows drive-like paths. Add a regression for Windows drive-like target names/paths, then reject : in path-safe descriptor names and descriptor-relative paths.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac2f95c0f5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/wesley-cli/src/main.rs

Copy link
Copy Markdown
Owner Author

Activity Summary

ID Severity Source File Commit Outcome
#705 P1 Issue crates/wesley-cli/src/main.rs, crates/wesley-cli/tests/cli.rs, docs/reference/cli.md, docs/reference/external-target-protocol.md ac2f95c0 Added wesley target verify <descriptor> [--json], deterministic descriptor diagnostics, non-execution coverage, parse-error coverage, docs, and changelog.
SELF-707-1 / discussion_r3524749983 P2 Self + PR review crates/wesley-cli/src/main.rs, crates/wesley-cli/tests/cli.rs, docs/reference/external-target-protocol.md 9368f48c Added RED/GREEN regression for Windows drive-like target paths, rejected : in descriptor names/paths, documented cross-host path rule, and resolved the review thread.
PR metadata P5 Self PR body n/a Updated Backout and Testing sections to cover both commits and the drive-path regression.

Verification:

  • cargo test -p wesley-cli --test cli target_verify_reports_descriptor_parse_errors RED before fix, GREEN after fix
  • cargo test -p wesley-cli --test cli target_verify_rejects_windows_drive_like_paths RED before fix, GREEN after fix
  • cargo test -p wesley-cli --test cli target_verify
  • cargo test -p wesley-cli --test cli
  • cargo fmt --check
  • pnpm exec prettier --check docs/reference/cli.md docs/reference/external-target-protocol.md CHANGELOG.md
  • cargo xtask docs-check
  • git diff --check
  • pnpm run preflight
  • pre-push Rust product preflight after each push
  • gh pr checks 707 all pass

Review state:

  • Review threads: resolved
  • CodeRabbit: required check passing; visible review-limit notice remains informational
  • Merge state: blocked only by review policy, so admin merge is eligible under the requested workflow.

@flyingrobots
flyingrobots merged commit a9acea3 into main Jul 5, 2026
20 checks passed
@flyingrobots
flyingrobots deleted the cli/target-descriptor-verify branch July 5, 2026 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cli: add external target descriptor verify

1 participant