docs: pre-release accuracy audit ahead of v1.7.0 - #288
Merged
Conversation
Three parallel audit passes found concrete drift between docs/ and the actual code, none of it caught by check_docs_guardrails.py (structural link/budget checks only, not content): - project-structure.md / architecture.md didn't mention core/sandbox/ (#266), web/runs.py (#262), web/i18n.py (#264), or core/stats.py (#268) at all; architecture.md still called SandboxRunner "future work (issue #157)" after it had already shipped. - configuration.md's "full parameter list" was missing glossary_store/ glossary_missing_queue. - grader-workflow.md never mentioned --stats/--stats-summary anywhere. - installation.md's pinned ruff>=0.4 vs pyproject.toml's actual >=0.15.19. - server-mode.md's SandboxRunner network guarantee read unconditional; the Windows exception was only in a separate paragraph below it. - SECURITY.md's web-shell scope bullet named no actual mechanism (Host/ Origin guard, path-confinement) despite both being fully implemented and documented in docs/api.md. - CLAUDE.md's metrics table: test count (967) matched no known-good source — CHECKPOINT.md and docs/versions.md both say 784 for the same v1.6.0 snapshot; Python 3.14 listed as equal to 3.12/3.13 despite being experimental/ubuntu-only in the CI matrix (README already knew this). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Requested audit for documentation drift before preparing the v1.7.0 release. Three parallel Explore passes over docs/ vs. actual code found concrete inaccuracies — none of them caught by
check_docs_guardrails.py(that script only checks README budget + link validity, not content accuracy).docs/project-structure.md/docs/architecture.md: didn't mentioncore/sandbox/(feat(core): SandboxRunner MVP (Linux, bubblewrap/nsjail) по требованиям issue #157 #266),web/runs.py(feat(web): job model — POST /api/v1/runs + polling, синхронный /api/grade deprecated #262),web/i18n.py(refactor(i18n): каталог сообщений для веб-ответов — message_id вместо русских строк в viewmodels #264), orcore/stats.py(feat(cli): локальная opt-in статистика запусков (режимы, вердикты, ОС) #268) at all.architecture.mdstill describedSandboxRunneras "future work (issue security(server): описать ограничения network-off, temp dirs, quotas #157)" even though it shipped in feat(core): SandboxRunner MVP (Linux, bubblewrap/nsjail) по требованиям issue #157 #266 — added proper module-table rows + DAG edges + layer-diagram entries for all four.docs/configuration.md: "full parameter list" table was missingglossary_store/glossary_missing_queue(they exist inconfig.py, were only documented indocs/glossary.md).docs/grader-workflow.md: never mentioned--stats/--stats-summaryanywhere (only inconfiguration.md) — added a section matching the style of the existing--cache/--sandboxsections.docs/installation.md: pinnedruff>=0.4vs.pyproject.toml's actual>=0.15.19.docs/server-mode.md:SandboxRunner's "network unreachable" guarantee read unconditional in the bullet list; the Windows exception was only in a separate paragraph below — now flagged inline.SECURITY.md: the web-shell scope bullet named no actual mechanism despite Host/Origin guard (fix: security(web): проверить Host/Origin для localhost API #242) and path-confinement (--root/--no-root-confinement, feat(web): workspace root (--root) — конфайнмент путей всех эндпоинтов #261) being fully implemented — added a dedicated section, cross-linked todocs/api.md.CLAUDE.md's metrics table: test count (967) matched no known-good source —CHECKPOINT.mdanddocs/versions.mdboth say 784 for the same v1.6.0 snapshot (967 was simply wrong); Python 3.14 was listed as equal to 3.12/3.13 despite being experimental/ubuntu-only in the CI matrix —README.md's prose already correctly said so, just not the metrics table or the badge.Separate from and unrelated to #287 (the badge-commit bug fix) — this PR is docs-content only, no code changes.
Test plan
python scripts/check_docs_guardrails.py→ README within budget, all 403 local Markdown links resolvepython scripts/check_version_consistency.py→ consistentpython scripts/check_locale_guardrails.py→ consistentCHANGELOG.mdupdated under## [Unreleased]→### Docs🤖 Generated with Claude Code