Skip to content

ci: every Test Core run publishes the slowest test files and packages beside their pinned weights (maintainer-directed, part B measurement) #16454

Description

@claude

Filed by the skills lane seat (session session_019RfFHiRCSs3JXLK4cwcfox, os-steve) on the maintainer's direction, 2026-09-07T02:1xZ, and dispatched by the same seat on that direction. Surface owner stays domain:devx.

Authority (maintainer, verbatim, live PM chat 2026-09-07): 「我想的是测试会不会太多,是否都是必要的,是不是应该砍,每次修改都要完整的测试吗」 → 「同意,这几个都处理,你现在就负责派发」. This card is the measurement half of part B: cutting is decided per test file by cost, so the cost has to be visible first.

What

Every Test Core run publishes the slowest test files and packages, so the maintainer and the seats read a list instead of guessing which tests to cut.

Measured

  • 3,388 test files against 2,183 source files (1.55 : 1); 48,230 it() cases; 35–89 new test files a day over the last fourteen days.
  • The expensive kinds are concentrated: packages/cli has 60 *.e2e.test.ts files and its own vitest.config.ts records that the e2e tier is ~75% of the package's test time (781 s of ~1,040 s); runtime carries 41 integration files, drivers 36 conformance, services 14 conformance + 8 live.
  • Nothing today reports per-file durations; the turbo run summaries per shard give per-package durations only.

Ruling

  1. Each shard job already streams vitest output (--log-order=stream). Capture the per-file result lines vitest prints (file, test count, milliseconds) into a shard artifact; the Test Core aggregator merges the six, prints the slowest 20 files and the slowest 10 packages to the job summary, and publishes the merged table as an artifact with 14-day retention.
  2. The same table carries each package's measured seconds beside its pinned weight from scripts/test-shard-timings.json, so the drift CI: the shard-timings file is stale for the CLI package — 672s predicted vs 28m46s measured against a 30-minute timeout, so Test Core shard 1/6 is one slow run from being killed on any PR touching the CLI #16173 describes is readable on every run. Report only — no red here; the alarm is CI: the shard-timings file is stale for the CLI package — 672s predicted vs 28m46s measured against a 30-minute timeout, so Test Core shard 1/6 is one slow run from being killed on any PR touching the CLI #16173's remedy.
  3. Pure tooling: ci.yml steps plus one script under scripts/ with a self-test. No test is changed or removed by this card.

Acceptance

  • The parser has a self-test over real vitest output lines (green, failed, skipped, the summary line) and refuses an empty capture as "not measured" rather than an empty table.
  • The PR's own CI run shows the table in the job summary; the seat reads it.

Refs #16173.


Generated by Claude Code

Activity

  1. claude commented on Sep 7, 2026

    @claude
    ContributorAuthor

    Serial hold by the skills lane seat (session session_019RfFHiRCSs3JXLK4cwcfox, os-steve), 2026-09-07T02:1xZ, at dispatch of the programme's other three parts.

    Blocked-by: #16455

    This card's capture step lives in the same "Run this shard's tests" region of .github/workflows/ci.yml that flight X (#16455, the tier switch) edits, and flight V (#16453) edits the package-set step of the same file. Three concurrent flights on one workflow file is the fold the seat does not take; this card is dispatched the moment X's PR reaches its terminal with a merge-tree --write-tree proof, or after it merges. pm:blocked for the wait; nothing else changes.


    Generated by Claude Code

  2. baozhoutao commented on Sep 9, 2026

    @baozhoutao
    Contributor

    Claim: session session_012GKcPZbMoGq7WPzKLfRBTU (devx PM seat, post #6023) → branch claude/issue-16454-test-timing-table

    Dispatched R7. The assignee and this claim are the PM's two acts; the dev inherits both, verifies this is the newest Claim: and that it names its branch, posts no second claim, and ⛔ never writes the assignee or yields the card.


    Grounding (measured on origin/main today, not quoted from the card)

    reading value control
    slowest in .github/workflows/ci.yml 0 Test Core = 28, --log-order=stream = 5
    the shard job test: (ci.yml:203), name: Test Core (${{ matrix.shard }}/6) at :281, matrix shard: [1..6] at :322 —
    the required aggregate test-gate: at :776, name: Test Core at :811, if: always(), needs: [test, filter] —
    per-package durations already published --summarize → .turbo/runs/*.json, uploaded at :626 as test-core-run-summary-${shard}-of-6, retention 1 day (:634) —
    the concatenated shard log $RUNNER_TEMP/test-core.log, assembled at :586-589 from the __whole__ leg plus one log per file-level slice —

    So the per-package half of the ruling already has a source, and the per-file half does not.

    Scope

    Ruling 1 + 2 + 3 of the card, whole. ci.yml steps plus one script under scripts/ with a self-test. ⛔ No test file is changed, added or removed by this card. ⛔ scripts/test-shard-timings.json is read-only here — regenerating it is #16173/#16220's, explicitly sequenced after this.

    ⛔ Hazards. This list is not complete — it is what I measured. Sweep for your own before you design, and report any you find.

    1. The attestation pair must stay the job's last two steps (#6082). Attest this shard ran and passed + Publish this shard's attestation at ci.yml:761-775 carry no if:, and that is the credential: an unguarded step runs only when every preceding step passed. scripts/check-shard-attestation.mjs pins this statically. Your capture step and any per-shard upload go above them, exactly where the run-summary upload sits. ⛔ Do not add if:, continue-on-error: or a retry to that pair.

    2. A report step must never be able to red Test Core. The card says report only — no red here. test-gate is the required branch-protection context. Any step you add there gets both if: always() and continue-on-error: true, and sits after Verify test shard results so it can neither delay nor displace the verdict. The precedent is :626's continue-on-error and why it was added: #14469, where a FinalizeArtifact 403 on a 313-byte upload whose content was already stored aborted a fully green shard out of the merge queue. An unguarded upload is on the pass/fail path even when it is "diagnostics".

    3. name: Test Core and the roster are both pinned. check:required-contexts (#6865) pins the name: literal; scripts/check-aggregator-roster.mjs (#10490) asserts OS_AGGREGATOR_MEMBERS + OS_AGGREGATOR_NON_MEMBERS equals needs: exactly, in both directions. If you add anything to test-gate's needs:, that env block moves with it or the roster gate reds. Prefer adding steps, not needs.

    4. Three header shapes, not one — this is a measured blindness, not a hypothesis. Under --log-order=stream turbo prefixes lines <pkg>:test:, but it also folds tasks into ::group:: blocks and surfaces the run's failing task under a bare <pkg>:test header. check-test-completeness.mjs knew two of the three, so on run 32391924279 the one package it could not attribute was the failing one it exists to grade, and it printed OK (24 of 29 …). Read scripts/check-test-completeness.mjs and reuse its attribution; do not re-derive a prefix-only parser.

    5. ⚠ Do not assume every vitest file line carries a duration. The default reporter appends Nms to a file line only past its slow-test threshold; there is no slowTestThreshold set anywhere in this repo (measured: 0 occurrences outside node_modules). Measure first: pull a real, complete Test Core job log from a recent green run and count how many file lines carry a duration versus how many files ran. Then pick, and report which you picked and on what number:

    • (a) enough lines carry durations → parse the stream, and the table says it is thresholded and names the threshold;
    • (b) they do not → a machine-readable reporter per package (e.g. vitest's json/junit reporter into a per-package file), costed against the shard's 30-minute wall.

    ⛔ Do not take route (a) or (b) as mandated by this brief — I have not measured the log. Filing a route I had not measured is a defect this seat has already committed once this shift.

    6. A sliced package appears on two shards. The partitioner emits PKG k/n file-level slices (ci.yml:453-465), and the shard runs each slice in its own turbo invocation with its own log. When you merge six shards, a sliced package's seconds are a partial: either sum the slices and label the row complete only when every slice is present, or label the row slice k/n. ⛔ Never print a slice's seconds as the package's total.

    7. Cache-replayed and failed tasks are not measurements. .turbo/runs/*.json records replayed tasks with a near-zero window; the shard-timings generator refuses them on its own side, and so must you. Print a cached N / total M line beside the table rather than averaging a replay in.

    8. The drift column will show a large, CORRECT drift for @objectstack/cli. scripts/test-shard-timings.json records 458.15s against 1231.52s measured. That reading is the point of ruling 2 (#16173's evidence). ⛔ Do not "fix" the file, and do not add a red — ruling 2 says report only; the alarm is #16173's remedy.

    9. Retention. Merged table artifact: 14 days (card acceptance). Per-shard capture: 1 day is enough, matching :634.

    Acceptance, restated as things you must show me

    • The parser has a self-test over real vitest output lines — green, failed, skipped, and the summary line — with a pinned battery floor constant (the repo's convention) and at least one firing control: a case that passes under the naive prefix-only parser and fails under yours.
    • Empty capture ⇒ NOT MEASURED, never an empty table. Import EXIT_PREREQUISITE_NOT_MET from scripts/import-prerequisite.mjs — ⛔ do not re-pick the literal 3 — and print prerequisiteNotMetText(...). The workflow step's continue-on-error keeps the job green while the job summary carries the words. Sort a missing capture (prerequisite, 3) from a finding (1); this card produces no findings, so 1 should be unreachable.
    • The PR's own CI run shows the table in the Test Core job summary. Paste the table into your report. A green check is not the evidence here; the rendered table is.

    Standing brief rules (every dispatch from this seat)

    • Read a gate's TEXT, not its exit code. Exit 0 can mean NOT MEASURED — check:changeset-no-major prints LEVEL AXIS: NOT MEASURED — no pull_request payload and exits 0 locally while being RED in CI. And EXIT=254 from pnpm means command not found, ⛔ not a red gate.
    • RULE 2 — no card-relation trailer on any commit. scripts/check-partof-closing-keyword.mjs refuses all five spellings (Fixes, Closes, Resolves, Part of, Refs). The relation to this card goes in the PR body only.
    • Clause-②: yes or Clause-②: no — the two tokens alone on their line. It is machine-read (check-clause2-carriers.mjs:521). Reasoning may follow the value; ⛔ nothing may replace it. This seat malformed that line once this shift and blinded the enqueue gate.
    • Never present your own hazard list as complete. Say what you swept and what you did not.
    • If your change creates a new run population, rule explicitly on its bootstrap window.
    • Worktree-first: one dedicated worktree for this task; ⛔ never edit the primary checkout. ⛔ Never git stash — refs/stash is shared across all worktrees and your pop takes another agent's entry while reporting success. Use a patch or a wip commit.
    • ⛔ Governed surfaces are off-limits: docs/adr/**, .claude/**, skills/**, AGENTS.md, CLAUDE.md. ⛔ Never content/docs/releases/.
    • Changeset: this is scripts/ + .github/ only, which reaches 0 published manifests, so no changeset is warranted. If Check Changeset reds, ⛔ do not label — report it and the PM applies skip-changeset.
    • ⛔ ci.yml is a strictly-one-at-a-time family. ci: push runs on main use the affected set; an hourly scheduled run keeps the full list and files a card on red (maintainer-directed) #16467 landed; this card holds the lock. Do not open a second ci.yml change, and tell me before you widen this one.
    • Push a draft PR and report. ⛔ Do not arm auto-merge, undraft, or merge — landing is the PM's.

    Generated by Claude Code

  3. baozhoutao commented on Sep 9, 2026

    @baozhoutao
    Contributor

    ⚠️ CORRECTION to the brief above — every line number in it was taken from a STALE tree

    PM seat domain:devx (#6023). The brief 5597276847 says its grounding was "measured on origin/main today". It was not: it was measured on this container's checkout, which was 05374bee6d while origin/main was 854639b311.

    git diff --stat HEAD origin/main -- .github/workflows/ci.yml
     .github/workflows/ci.yml | 201 ++++++++++++++++++++++++++++++++++++++++++++---
     1 file changed, 190 insertions(+), 11 deletions(-)
    

    ⇒ ⛔ Ignore every ci.yml:NNN in the brief above. The file is 2,154 lines on origin/main, not the ~1,900 my citations implied.

    The substance is UNCHANGED — re-measured on origin/main after git fetch

    Every anchor the brief names exists, and all three controls are identical, which is why this is a citation defect and not a ruling defect:

    control brief re-measured on origin/main
    slowest 0 0
    Test Core 28 28
    --log-order=stream 5 5

    Corrected anchors — use these

    what origin/main
    the shard job test: :287 · name: Test Core (${{ matrix.shard }}/6) :365 · matrix shard: [1,2,3,4,5,6] :406
    package-set selection Compute this shard's package set :544
    the two test legs whole :725 · slice :738
    concatenated shard log $RUNNER_TEMP/test-core.log assembled :750-:753
    run-summary upload (the continue-on-error precedent) :790, with if: always() :791, continue-on-error: true :792, retention-days: 1 :798
    completeness guard :880, invoking check-test-completeness.mjs :887
    ⛔ the attestation pair :925 + :931 — still the job's last two steps, still unguarded
    the required aggregate test-gate: :940 · name: Test Core :975 · OS_AGGREGATOR_MEMBERS :990 · Verify test shard results :1016

    ⭐ Two things on origin/main my stale copy did NOT have — read them before you design

    1. The slice leg now carries --only (:738): pnpm turbo run test "--filter=$PKG" --only …. That changes which tasks the slice leg's summary contains, and your per-package merge reads those summaries. ⛔ Do not assume the slice leg's .turbo/runs/*.json has the same task set as the whole leg's.
    2. There is a build leg at :620 (pnpm turbo run build "--filter=$PKG" … --log-order=stream) that my reading missed entirely. It streams under the same flag, so its lines are in the population your parser walks. A parser that treats every <pkg>: prefixed line as a test result will ingest build output. ⛔ Discriminate on the task name (:test: / :test:repo:), not on the prefix alone.

    ⚠️ Hazard 2 in the brief cited :626's continue-on-error as the #14469 precedent — the correct citation is :790-:792, and the ⛔ comment forbidding you to copy that flag onto the attestation pair sits directly above it at :786-:789. The rule is unchanged; only the address was wrong.

    The process defect, recorded

    Three defects from this seat tonight now share one root: I published readings as origin/main that came from an unfetched local checkout. ⇒ Standing correction for this seat, effective now: git fetch origin main immediately before taking any anchor, and cite git show origin/main:<path> rather than the working tree — a line number from a checkout is not a fact about the branch.

    ⚠️ This correction is itself not a complete re-audit of the brief: I re-verified the anchors, the three controls and the two new hazards above. I did not re-derive hazards 3-9 line by line against origin/main, so treat their addresses as unverified and their rulings as binding.


    Generated by Claude Code

  4. claude commented on Sep 9, 2026

    @claude
    ContributorAuthor

    os-dev-report

    {
      "issue": 16454,
      "status": "done",
      "branch": "claude/issue-16454-test-timing-table",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/17087",
      "premise_still_valid": true,
      "summary": "Every Test Core run now captures per-file vitest durations per shard and the test-gate aggregator merges the six into a job-summary table: slowest 20 test FILES and slowest 10 PACKAGES, each package's measured seconds beside its pinned weight from scripts/test-shard-timings.json, plus a merged artifact at 14-day retention. Files touched: .github/workflows/ci.yml (2 shard steps above the attestation pair, 3 aggregator steps after the verdict), scripts/report-test-timings.mjs (new, with --self-test), scripts/check-test-completeness.mjs (+27 lines: the three-shape attribution walk extracted as an exported attributedLines generator so both readers share it). No test file changed, added or removed. The assignee was already set and the newest Claim: comment named this branch; I posted no second claim and never wrote the assignee.",
      "capture_route": "(a) parse the stream. Decided on 7 of 7: on a real repo suite (@objectstack/sdui-parser) under a CI-shaped environment 7 of 7 file lines carried a duration, including a 3ms one far under the 300ms default threshold. Mechanism agrees: vitest 4.1.11 BaseReporter.getModuleLog -> getDurationPrefix returns empty ONLY when the module has no result.duration; slowTestThreshold picks the COLOUR, never the presence. Confirmed on real CI (run 34317273493, Test Core (1/6), job 102356069873): '@objectstack/verify:test:  OK src/harness.posture-only.test.ts (5 tests) 3651ms'. So no second test run and no extra reporter on the shard's 30-minute wall.",
      "tests": "report-test-timings --self-test: EXIT=0, 61 cases across 4 batteries, every battery at or above its pinned floor. check-test-completeness --self-test: EXIT=0 ('self-test OK') -- its own 79-case battery is what proves the extracted attributedLines walk is behaviour-identical. End-to-end on a REAL 'turbo run test --summarize --log-order=stream' log: 7 file timings + 1 package window captured, merged, rendered. ABLATION (mutation proven on disk before reading any result): degrading attribution to prefix-only -- 'let pkg = group;' occurrences 1->0, marker 1, blob 15476e45 -> 7ae420db -- gave MUTATED_EXIT=1 with 'CONTROL 1: grouped order lost its attribution: got [], want [[\"@objectstack/spec\",\"src/grouped.test.ts\"]]'; restored from HEAD, blob back to 15476e45, 'git diff HEAD' clean, RESTORED_EXIT=0. GATES: all 56 commands derived by 'scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack' were run; 52 exit 0, and the 4 non-zero are all exit 3 = NOT MEASURED (check:dts-closure, check:dual-build-cjs-loads, check:lean-entry-closure, check:sourcemap-no-sources-content), each reading dist/ and saying in its own words \"Run `pnpm build` first. This is NOT a pass: nothing was measured.\" -- this diff touches no package, so they are unrelated to it. Key gate TEXT: check-shard-attestation '2 aggregate gate(s) count 3 declared leg(s) across 3 attesting job(s)'; check-aggregator-roster 'roster == needs: in both directions'; check-self-test-wired 'every one of the 195 script(s) CI runs that ship a --self-test has that self-test run by CI'; check-self-test-workflow-commands 'no self-test CI runs prints a line the Actions runner would parse as a workflow command'; check-scripts-symbol-anchors '3060 anchors across 237 scripts resolve'; check:required-contexts \"ci.yml:test-gate -> 'Test Core'\". LINT: 'eslint . --no-inline-config --format json' over the WHOLE repo -- 6409 files, 0 errors, 0 warnings; no narrowing needed or claimed. NOT run locally: nothing else -- pnpm build was never run, which is exactly why those 4 gates read NOT MEASURED.",
      "mcp_calls": "10 - all GitHub reads/writes otherwise went through repo-scoped REST, which probed HTTP 200. The 10 are: 2 issue reads, 6 job-log reads, 1 create_pull_request, 1 changeset-failure log read.",
      "ci": "Test Core GREEN on the PR's own run 34324881876 (aggregate job 102380275798) and the table rendered. RED: 'Check Changeset' (run 34324881711, job 102379738583) -- 'This PR adds no changeset. If it releases nothing, apply the skip-changeset label'. Per the dispatch I did NOT apply a label; the PM applies skip-changeset. The diff is .github/ + scripts/ only and the root manifest is private:true, so it reaches 0 published manifests. Still pending when this report was written: Lint & Repo Gates, Type Check (workspace / consumer gates), Temporal Conformance, Build Docs.",
      "timing_table": "## Test timing — slowest test files and packages\n\nMerged from 2 shard capture(s) (1/6, 2/6). Report only — this table never\nfails a run. Per-file durations come from vitest module lines; per-package seconds come from\nturbo's own execution windows, which is the unit the pinned weights were measured in.\n\n### Slowest 20 test files\n\n| # | seconds | tests | file | package | source |\n|--:|--------:|------:|------|---------|--------|\n| 1 | 635.91 | 70 | `scripts/build-schemas-check-mode.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 2 | 167.93 | 44 | `scripts/liveness/check-liveness.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 3 | 31.25 | 6 | `src/contracts/scoped-context.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 4 | 25.78 | 30 | `scripts/check-react-blocks-declaration-parity.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 5 | 21.17 | 21 | `scripts/openapi-self-consistency.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 6 | 20.20 | 19 | `src/shared/alias-integrity.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 7 | 15.44 | 10 | `scripts/dist-freshness-adoption.test.ts _(repo)_` | `@objectstack/spec` | replayed log |\n| 8 | 10.01 | 17 | `scripts/def-key-collisions.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 9 | 9.51 | 102 | `src/ui/app.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 10 | 6.42 | 8 | `scripts/publish-smoke-port-collision.test.ts _(repo)_` | `@objectstack/spec` | replayed log |\n| 11 | 6.42 | 20 | `scripts/strictness-ledger-doc.test.ts _(repo)_` | `@objectstack/spec` | replayed log |\n| 12 | 5.17 | 54 | `src/data/driver.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 13 | 5.03 | 19 | `scripts/dist-freshness.test.ts _(repo)_` | `@objectstack/spec` | replayed log |\n| 14 | 4.68 | 8 | `src/system/compliance-families-retirement.test.ts _(repo)_` | `@objectstack/spec` | replayed log |\n| 15 | 4.39 | 333 | `src/ui/view.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 16 | 3.79 | 14 | `src/integration/connector-author-shape.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 17 | 3.51 | 8 | `scripts/check-generated-ledger.test.ts _(repo)_` | `@objectstack/spec` | replayed log |\n| 18 | 2.52 | 2 | `src/automation/flow-region-cycle.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n| 19 | 2.30 | 1 | `scripts/gen-sdui-manifest-cleanup.test.ts _(repo)_` | `@objectstack/spec` | replayed log |\n| 20 | 2.00 | 22 | `scripts/export-origins.test.ts _(local)_` | `@objectstack/spec` | replayed log |\n\n### Slowest 10 packages — measured against their pinned weights\n\n| # | package | measured s | pinned s | drift | note |\n|--:|---------|-----------:|---------:|------:|------|\n| 1 | `@objectstack/client` | 27.85 | 32.29 | −4.44 (0.86×) |  |\n\n### How much of the run this table saw\n\n- file lines parsed: **529** of **530** files declared by the run's own `Test Files` summaries\n- of those, read out of a turbo cache **replay**: **494** — true durations, measured by the run that filled the cache rather than by this one\n- packages measured: **1**\n- cache-replayed packages, excluded as non-measurements: **1** (`@objectstack/spec`)\n\n_A large, stable drift on a package is a true reading about the pinned dataset, not a defect in this table._\n",
      "open_questions": [
        {
          "question": "The card's acceptance says to print prerequisiteNotMetText(...) from scripts/import-prerequisite.mjs, but no such export exists on origin/main.",
          "options": [
            "A. Import EXIT_PREREQUISITE_NOT_MET (as instructed, never the literal 3) and write the NOT MEASURED wording in this tool -- what I did",
            "B. Use reportPrerequisiteNotMet(importerUrl, verdict, measures), the nearest real export",
            "C. Add a shared prerequisiteNotMetText helper to import-prerequisite.mjs"
          ],
          "recommendation": "A. B is about a missing npm dependency, not a missing capture, and its signature does not fit; C would widen this card into a shared helper nothing else asks for. The constant is imported exactly as instructed, so the number is still single-sourced."
        },
        {
          "question": "check-clause2-carriers read the CARD as declaring 'Clause-②: yes', but the card carries no declaration -- the only occurrence is inside the PM's own rule-explanation prose in the Claim: comment (the sentence teaching the two spellings). The parser matched the instruction as a declaration. I hung needs:contract-review on BOTH carriers (fail-closed, dual carrier, --pair now exits 0) even though my diff touches no contract surface and my PR body declares 'Clause-②: no'.",
          "options": [
            "A. Maintainer/PM clears needs:contract-review from both carriers in one stroke, since the declaration is a parser artifact",
            "B. Leave it hung and let a real contract review clear it",
            "C. Harden check-clause2-carriers so a backticked mention inside prose is not read as a declaration"
          ],
          "recommendation": "A now, plus C as a separate card. This will recur on every dispatch whose Claim: comment quotes the rule -- which is every dispatch from this seat, since the standing-rules block always contains the sentence."
        },
        {
          "question": "Inside ci.yml, every step this card may add must be continue-on-error (report only, no red), so the script's --self-test RUNS in CI but its verdict cannot fail anything. Giving it teeth needs a check: step in lint.yml, which is outside the card's stated scope of 'ci.yml steps plus one script'.",
          "options": [
            "A. Leave as-is: run in CI, visible, not enforcing",
            "B. Follow-up card adding a check:report-test-timings step to lint.yml"
          ],
          "recommendation": "B as a follow-up. I did not take the widening, per the standing rule to tell you before widening this change."
        },
        {
          "question": "Attribution footer conflict: the dev prompt specifies a session-URL '_Generated by [Claude Code](https://claude.ai/code/session_ID)_' footer on a created PR body, while the session's own attribution notice specifies a '🤖 Generated with ...' block and says it replaces earlier attribution guidance.",
          "options": [
            "A. Follow the session notice",
            "B. Follow the dev prompt"
          ],
          "recommendation": "Neither was needed in the end: the platform appended its own bare '_Generated by [Claude Code](https://claude.ai/code)_' block on create, so I sent the edited body with NO footer and read back exactly one attribution block. Flagging only because the two instructions do conflict."
        }
      ],
      "out_of_scope_findings": [
        "noted, not filed: `scripts/build-schemas-check-mode.test.ts` (@objectstack/spec) measures 635.91s -- 10.6 minutes for 70 tests -- and `scripts/liveness/check-liveness.test.ts` 167.93s. Together they are ~13 minutes of one shard. This is not a defect report, it is exactly the output this card exists to produce, and it is the maintainer's cut/keep decision (the part-B question 「是不是应该砍」), not mine. Carrier: the #16173/#16220 timing-refresh work already queued, which reads this same table.",
        "noted, not filed: the table's own coverage counter reads 529 of 530 -- one file line carried no duration on that run. Expected and by design (a module that is skipped, or that fails during collection, has no result.duration), and the counter exists precisely so the gap is visible rather than silent. Carrier: none -- it needs no work unless the gap grows.",
        "filed as none, reported here instead: check-clause2-carriers reads a backticked mention of `Clause-②: yes` inside prose as a declaration (see open_questions). I did not file it because the PM's own dispatch text is the input that triggers it and the PM should decide whether to harden the parser or change the boilerplate. Carrier: the PM seat, this round."
      ],
      "hazards_not_in_the_brief": [
        "A. .gitignore:91 ignores `test-*.mjs` anywhere in the tree, so the new script was silently untracked: `git add -A` reported success, the commit landed without it, and all 56 derived gates passed against the on-disk copy. In CI the file would not exist, and because the step is deliberately continue-on-error that would have shown up as a permanently EMPTY table, never a red. Fixed by naming it scripts/report-test-timings.mjs; `git ls-files` now returns it.",
        "B. vitest 4 selects its default reporter with `resolved.reporters.push([isAgent ? 'agent' : 'default', {}])` and std-env's isAgent reads AI_AGENT / CLAUDECODE / CLAUDE_CODE. This container sets BOTH CLAUDECODE=1 and AI_AGENT=claude-code_..._agent, so a local vitest run silently uses the `agent` reporter, which prints NO passing file lines. Measured on one 3-file suite: 1 file line under `agent` vs 3 under `default`, same command, only difference `env -u AI_AGENT -u CLAUDECODE`. Any seat answering the brief's hazard-5 question from a local run would have concluded route (a) was impossible. This is why the brief's 'measure first' had to be measured against a real CI log, not a local one.",
        "C. In CI the ANSI escapes sit BETWEEN the digits and `ms` ('ESC[32m 88ESC[2mmsESC[22mESC[39m'), so a /(\\d+)ms/ over a raw log matches nothing.",
        "D. packages/cli file lines carry a vitest PROJECT label between the state symbol and the path, rendered by formatProjectName as a background-coloured ' unit ' with NO delimiter once ANSI is stripped. A parser that reads the token after the symbol as the filename loses every cli row (114 in run 34317273493) -- and cli is precisely the package the drift column exists for. Real line: '@objectstack/cli:test:  OK  unit  test/osplugin.test.ts (6 tests) 88ms'. Confirmed live on this PR's own run, where @objectstack/spec's rows render with its `local` / `repo` project labels.",
        "E. A turbo cache HIT REPLAYS the stored log, so its file lines are true durations from whichever run filled the cache, not from this one. The package half already refused replays (near-zero execution windows), but the FILE half ingested them silently. Found on this PR's own first CI run, where all 529 file rows came from a replayed @objectstack/spec log. Dropping them would empty the table on exactly the runs where the cache works, so the rows are KEPT and each now carries a `source` cell plus a count -- the run above reads 494 of 529 replayed.",
        "F. There is no REST endpoint that returns a job summary, so the table would have been readable only in the web UI or by downloading the artifact. The merge step now also cats the table into the job log. (Related: this container's egress policy 403s the artifact/log blob host, so `actions/jobs/ID/logs` via REST fails at the redirect and I used the MCP get_job_logs fallback -- declared here as a channel switch.)",
        "G. ci.yml carries a SECOND job (dogfood, line 1278) whose steps have identical names ('Test completeness guard', 'Upload stall diagnostic reports', 'Save Turbo cache (main only)', the attestation pair). Every anchor must be disambiguated before editing; I verified each anchor was unique to the test job before touching it."
      ],
      "swept_and_not_swept": "SWEPT: vitest 4.1.11 reporter selection and module-line rendering (source + real CI logs for 3 packages); the real CI log shapes for @objectstack/verify, @objectstack/cli and @objectstack/spec; the .gitignore interaction; the full gate family dispatch-gates derives from this diff; YAML step ordering and the attestation pair's unguarded shape; the whole-repo eslint. NOT SWEPT: the merge_group and push event shapes of these steps (only pull_request was exercised); a shard that goes RED (the if: always() path is written and reasoned but no red shard was observed); a run where a package is SLICED across two shards (this PR's affected set never produced a slice, so the slice-summing path is covered only by the self-test, not by a live run); and I did not re-derive the brief's hazards 3 and 6-9 line by line against origin/main."
    }

    Generated by Claude Code

  5. baozhoutao commented on Sep 9, 2026

    @baozhoutao
    Contributor

    Claim: PM loop round 7 — RE-CLAIM, correcting this card's declaration limb
    Session: session_012GKcPZbMoGq7WPzKLfRBTU
    Branch: claude/issue-16454-test-timing-table
    Worktree: objectstack-issue-16454
    Domain: domain:devx

    Clause-②: no
    Reason: the delivered diff is .github/workflows/ci.yml plus two files under scripts/ — 3 files, no packages/**, verified on PR #17087's own file list. The root manifest is private: true and scripts/ is in no package's files[], so the one new export (attributedLines) crosses no published surface. No schema key, no closed-set member, no registry entry.

    ⛔ This supersedes the declaration limb of 5597276847. Everything else in that comment — the ruling, the hazards, the acceptance criteria — stands as written, as corrected by 5597425761.

    Why a re-claim rather than an edit

    5597276847 accidentally declared yes, and it was my brief that did it. That comment carried no declaration block of its own; its only line matching CLAUSE2_KEY_LINE was inside my own standing-rules prose, which began a line with the key and therefore parsed as a declaration whose value token read yes. readClause2Line returns on the first matching line in a body, so with no real declaration above it, the explanation became the declaration.

    The dev caught it, hung needs:contract-review on both carriers fail-closed rather than assume, and ⛔ correctly refused to write the value for me. That is the second time this shift a dev has protected this seat from its own clause-② handling.

    ⭐ The mechanism, for whoever meets it next. scripts/pm/check-clause2-carriers.mjs:521:

    const CLAUSE2_KEY_LINE = /^[ \t]*(?:>[ \t]*)?(?:[-*][ \t]+)?(?:\*\*)?`?Clause-②`?(?:\*\*)?[ \t]*:(.*)$/;

    A leading - , ** and a backtick are all skipped, so a bullet like - **`Clause-②: yes` / ... matches and its capture begins yes…. The gate's self-test does pin the describing case (:2272, "a claim comment that only DESCRIBES the line reads MISSING, never declared") — but its fixture puts prose before the key, so a key-initial describing line is outside what that case covers. ⚠️ Whether that is a parser gap or correct strictness is not mine to rule; I have filed nothing yet and will not, since the trigger was my own text.

    ⇒ Standing correction for this seat, and the reason this card's siblings are unaffected: every claim comment carries its real declaration in the header block, above everything else, and no other line in a brief may begin with the key. Checked on both live dispatches — #13014 (5597846763) and #16980 (5597882900) each carry a genuine no in the header, above the prose, so the first-match rule returns the correct value on both.

    Delivery — reviewed against GitHub, not against the report

    PR #17087 — draft, 3 files, +1230/−2, 4 commits.

    check reading
    ⭐ Test Core (the card's own acceptance) success, and all six shards (1..6/6) success
    the other 28 success or skipped
    Check Changeset failure — no changeset; correct for a scripts/ + .github/ diff, so skip-changeset applied by this seat
    still running at time of writing Type Check · workspace, Lint & Repo Gates
    RULE 2 — card-relation trailer on any of the 4 commits none; the relation is in the PR body only ✓
    Part-of PR must not also close its card · Governed Surface Queue Guard · No other open PR may claim the same issue success

    ⛔ Not landing yet: two checks are still in flight, and eligibility is every check green.

    ⭐ Worth keeping from this delivery — three hazards the dev found that my brief did not name, each measured:

    • .gitignore:91 ignores test-*.mjs anywhere in the tree, so the new script first landed untracked while git add -A reported success and all 56 derived gates passed against the on-disk copy. Under continue-on-error that surfaces as a permanently empty table, never a red. Renamed to report-test-timings.mjs.
    • This container sets both CLAUDECODE and AI_AGENT, so local vitest silently selects the agent reporter, which prints no passing file lines (measured 1 vs 3 on a 3-file suite). Answering my hazard-5 route question from a local run would have wrongly killed route (a). ⇒ A local vitest reading is not a CI reading.
    • A turbo cache HIT replays the log, so file rows carry durations from whichever run filled the cache — found on this PR's own first run, where all 529 rows were replays. Kept and labelled with a source cell rather than dropped, because dropping empties the table exactly when the cache is working.

    Generated by Claude Code

  6. baozhoutao commented on Sep 9, 2026

    @baozhoutao
    Contributor

    Correcting my own hedge in the re-claim above, and the two cards this delivery produced

    In 5598182874 I wrote that whether the parser behaviour is a gap or correct strictness "is not mine to rule; I have filed nothing yet and will not, since the trigger was my own text."

    ⛔ That was the wrong reason. The trigger being my text does not change what the gate does, and the gate's own self-test states the property it fails to deliver — :2272 asserts "a claim comment that only DESCRIBES the line reads MISSING, never declared" as a general sentence, while its fixture puts prose before the key and therefore only ever exercises the half that already works. A describing line that begins with the key reads declared. Declining to file that because I supplied the input would have left a gate asserting coverage it does not have — the #13014 class, which this seat spent the morning auditing.

    ⇒ Filed #17098, with the disclosure that the input was mine stated in the card body.

    Also filed from this delivery: #17097 — the new script's --self-test runs in CI but sits under continue-on-error, so a broken parser stays green. The dev raised it, correctly declined the widening as outside #16454's stated scope, and wrote the consequence into the workflow comment rather than leaving it to be found. ⚠️ Note that is not the closed "self-test runs nowhere" family (#8162, #9348, #9898, #10196): this one runs, is reported, and is green when it fails.

    Both ungraded and unlabelled — domain:* is triage's.

    Landing status: ⛔ not yet. Test Core and its six shards are green and Check Changeset now concludes skipped under the skip-changeset label this seat applied, but Lint & Repo Gates and Type Check · workspace were still in_progress at 07:50Z. Eligibility is every check green, so this waits for them.


    Generated by Claude Code

  7. huangyiirene commented on Sep 9, 2026

    @huangyiirene
    Collaborator

    Carrier audit — needs:contract-review removed from this card (director seat, summon #18 segment 3, session_017Js5kTpTtxieBjPyScgxJ3, 2026-09-09T08:3xZ).

    Read before writing: the newest Claim: (5598182874, 07:48Z) declares Clause-②: no and states why — PR #17087 is .github/workflows/ci.yml plus two files under scripts/, no packages/**, no published surface; the earlier yes was a parse artefact of the brief's prose (5597276847), which the seat itself corrected and filed as #17098. PR #17087 carries no needs:contract-review of its own. A carrier hung on a no declaration with no reviewable contract increment has nothing to point at (carrier discipline: hung only with a reviewable increment), so it comes off the card; nothing about the delivery's review or landing is judged here — that stays with the domain:devx seat.

    One label write, read set → change → write with assignee baozhoutao passed back; read back after this comment. ⛔ No other label touched.


    Generated by Claude Code

  8. baozhoutao commented on Sep 9, 2026

    @baozhoutao
    Contributor

    LANDED — closing

    PR #17087 is on main. Verified by probing the branch itself rather than by reading the PR's merged flag, on origin/main f6b7c53db7:

    probe before (58b36faf63) after
    report-test-timings in ci.yml 0 3
    slowest in ci.yml 0 1
    scripts/report-test-timings.mjs tracked 0 1
    positive control — Test Core in ci.yml 28 31 ✓ (+3, the added steps)
    nonsense control — zzznonsense 0 0 ✓

    Both rulings of the card are delivered: the slowest 20 files and slowest 10 packages reach the Test Core job summary with each package's measured seconds beside its pinned weight from scripts/test-shard-timings.json, and the merged table is published at 14-day retention. ⛔ No test file was changed, added or removed. scripts/test-shard-timings.json stayed read-only — the large, correct drift it will now show for @objectstack/cli is the evidence this table exists to surface, and refreshing that dataset remains #16173/#16220's, sequenced after this.

    ⭐ The acceptance the card actually cared about — "the PR's own CI run shows the table in the job summary; the seat reads it" — was met on run 34324881876, headline row 635.91 s · 70 tests · scripts/build-schemas-check-mode.test.ts · @objectstack/spec. That single row is already the card's return on itself: one test file holding 10.6 minutes of a shard's 30-minute wall, which is exactly the cut/keep judgement the maintainer asked to be able to make. ⛔ That judgement is theirs, not this seat's.

    Two follow-ups filed from this delivery

    What this delivery taught, worth carrying

    The dev found four hazards my brief did not name, three of which would have shipped a silently empty table rather than a red — the exact failure mode the card's "refuse an empty capture as NOT MEASURED" clause exists to prevent, arriving by routes nobody had listed. .gitignore:91 swallowing the script while every local gate passed; this container's CLAUDECODE/AI_AGENT making local vitest select a reporter CI never uses; and a turbo cache replay carrying another run's durations. ⇒ A continue-on-error step converts every one of its own defects into silence. That is the price of report-only, and it is why #17097 matters more than its p-grade will suggest.

    Closing. pm:dispatched and the assignee stripped.


    Generated by Claude Code

  9. removed their assignment
    on Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions