Skill Validation Skill

An evidence-based, native Rust completion gate for Codex skills and Agent workflows. It turns a requirements contract into a machine-generated JSON report, so an Agent cannot legitimately claim completion merely because it says “done.”

中文说明

What it provides

The Cargo workspace is split by function and runs independent requirements in parallel with Rayon. It uses native parsers and deterministic evidence wherever practical; it returns unknown when a fact cannot be established safely.

Crate	Responsibility
validation-core	Contract types, 80+ check variants, statuses, evidence, and path handling.
validation-generic	Files, directories, text, regex, JSON Pointer, SHA-256, and adapter reports.
validation-code	Build/test reports, coverage, API/file symbols, dependency risk, and benchmarks.
validation-pptx	PPTX OOXML slides, named elements, bounds, images, hashes, and text.
validation-paper	Paper structure for Markdown, LaTeX, DOCX, and PDF.
validation-web	HTML elements/routes, form reports, screenshot diffs, console reports, and accessibility reports.
validation-spreadsheet	XLSX sheets, headers, formulas, ranges, blank ratios, and charts.
validation-data	CSV/JSON schemas, primary/foreign keys, row counts, outliers, reconciliation, and SQLite queries.
validation-document	DOCX/PDF/Markdown/LaTeX page, heading, table, signature, clause, forbidden-term, and TOC checks.
validation-image	Raster format/dimension/alpha/hash checks and SVG/design layer or naming checks.
validation-media	ffprobe-style duration/stream/resolution reports, subtitles, silence, loudness, transcripts, and frames.
validation-api	Status, JSON Schema, fields, latency, retries, idempotency, and side-effect reports.
validation-research	Source count, link accessibility evidence, years, claim coverage, and traceability.
validation-ops	Terraform/Kubernetes/YAML structure, deployment health, image tags, environment, metrics, and rollback.
validation-compliance	Licenses, privacy fields, brand terms, versions, release archives, and changelogs.
validation-agent	Step outputs, artifact consumption, dependency order, and verifier-generated final reports.
skill-validation-cli	Parallel orchestrator and skill-validation command-line binary.

Build

Requires a recent stable Rust toolchain.

cargo build --release

The binary is target/release/skill-validation.

Run a gate

target/release/skill-validation verify \
  --contract examples/all-scenarios-contract.json \
  --output work/verification.json

For a checked-in, runnable integration fixture covering the remaining validator families:

# Optional fixture regeneration; the deterministic fixture files are checked in.
python3 examples/full-integration/generate-fixtures.py
target/release/skill-validation verify \
  --contract examples/full-integration/preflight-contract.json \
  --output examples/full-integration/generated/preflight-verification.json
target/release/skill-validation verify \
  --contract examples/full-integration/requirements.json \
  --output examples/full-integration/generated/verification.json

Exit codes:

• 0: every required requirement passed;
• 1: at least one required requirement is fail or unknown;
• 2: contract, input, or runtime error.

Completion is allowed only when the report contains:

{
  "completion_allowed": true,
  "overall_status": "pass"
}

All paths are resolved relative to the contract file. Each requirement is required by default, and every check inside a requirement must pass.

Discovery commands

Inspect real artifacts before writing identifiers into a contract:

target/release/skill-validation inspect-pptx output/deck.pptx
target/release/skill-validation inspect-paper output/paper.tex
target/release/skill-validation inspect-spreadsheet output/model.xlsx
target/release/skill-validation inspect-document output/contract.pdf
target/release/skill-validation inspect-media output/video.mp4

The media inspector calls a local ffprobe executable. Browser, accessibility, security-scanner, link-checker, CI, and deployment facts should be emitted as JSON reports by the corresponding tool, then checked by Rust.

Examples

• examples/generic-contract.json: files, text, hashes, and JSON values.
• examples/pptx-contract.json: slide count, named PPTX object, image, bounds, and text.
• examples/paper-contract.json: sections, words, citations, references, figures, tables, and placeholders.
• examples/all-scenarios-contract.json: one contract covering every validator family.
• examples/pptx-github-footer/: an end-to-end PPTX fixture, deterministic image-evidence adapter, and verification runbook.
• examples/full-integration/: a runnable integration fixture for 14 non-PPTX validator families, 87 checks, and verifier-proven agent completion; its coverage audit combines with the PPTX fixture to execute all 92 CheckSpec kinds.
• references/verification-contract.schema.json: JSON Schema for the contract envelope.
• references/adapter-authoring.md: evidence rules for a custom Rust adapter.

The comprehensive contract expects evidence files such as browser reports, ffprobe output, CI coverage, dependency audits, and deployment monitor results. It is a template, not a claim that those sample paths already exist.

Trust boundary

This skill prevents accidental hallucinated completion claims. It is not a security boundary if the same Agent can modify the artifact, verifier, and report. For compliance-sensitive work, run the release binary in protected CI or an external read-only environment.

中文说明

这是一个完全由 Rust 实现的 Codex skill / Agent 交付门禁。它将需求契约转换成机器生成的 JSON 验证报告，要求模型只有在 Rust 验证器确认所有必需项后才能宣布完成。

已覆盖的场景

• 代码：构建、测试覆盖率、指定 API/文件、依赖高危漏洞、性能基准；
• Web/App：HTML 元素、路由、表单、截图回归、控制台错误、可访问性报告；
• Excel/XLSX：工作表、列名、公式、范围值、空值率、图表；
• CSV/JSON/SQLite：Schema、主键、外键、行数、异常值、跨表对账、数据库查询；
• Word/PDF/合同：页数、标题层级、表格、签名页、关键条款、敏感词、目录一致性；
• 图片/设计：格式、尺寸、透明度、SHA-256、命名规则、SVG/设计图层；
• 视频/音频：时长、分辨率、字幕轨、静音、响度、字幕台词、关键帧报告；
• API/自动化：状态码、JSON Schema、字段、幂等、延迟、重试、副作用；
• 研究/检索：来源数量、链接可访问性、年份、结论覆盖、引用可追溯；
• 运维/IaC：Terraform/Kubernetes 配置、部署健康、镜像 tag、环境变量、监控指标、回滚；
• 合规/发布：许可证、隐私字段、品牌术语、版本一致、发布包、变更日志；
• Agent 工作流：步骤输出、下游消费、依赖顺序、最终验证报告来源。

架构与性能

不同功能位于独立 Rust crate；CLI 使用 Rayon 并行执行相互独立的 requirement，并在 XLSX、PPTX、论文和文档检查中缓存索引。PPTX/XLSX/DOCX 使用 ZIP/OOXML 原生读取，SQLite 使用 rusqlite，JSON Schema 使用 jsonschema，图片使用 image。

外部工具的事实仍然必须先形成证据报告。例如浏览器截图、axe 可访问性、ffprobe 媒体信息、CI 覆盖率、依赖审计、部署健康和链接检查结果由对应工具产生，再由本项目的 Rust 验证器确认。模型不能直接在 JSON 里把某一项写成 true。

构建与运行

cargo build --release
target/release/skill-validation verify \
  --contract examples/all-scenarios-contract.json \
  --output work/verification.json

要实际运行除 PPTX 之外的全部模块，可使用 examples/full-integration/：先生成确定性夹具，再依次运行 preflight、全量 Rust 验证和 check 覆盖审计。该目录覆盖 14 个模块、87 条 check；与 PPTX 示例合并后，92 种 CheckSpec 全部实际执行并通过，Agent 最终报告也必须由验证器生成。

只有所有必需检查都是 pass，并且生成报告中的 completion_allowed 为 true，才可以宣布完成。fail 或 unknown 都必须修复或明确交给人工复核。

信任边界

它用于阻止无意中的幻觉式完成声明；如果同一个 Agent 能同时修改产物、验证器和报告，它不是安全边界。对合规或对抗性场景，应在受保护 CI 或外部只读环境运行 release 二进制。