Skip to content

Bring all servers to the 90% per-file coverage gate; reintroduce the rule to AGENTS.md #4474

Description

@cliffhall

Goal

Bring every server up to the ≥ 90% per-file test-coverage gate used in the inspector repo, enforce it in tooling, and then reintroduce the rule into AGENTS.md — #4473 deliberately deferred it so the rule wouldn't be aspirational fiction on day one. Once this issue lands, the rule is true and stays true.

The gate (per the inspector v2/main reference)

  • Per-file threshold of ≥ 90 on all four dimensions: lines, statements, functions, branches
  • Genuinely-unreachable branches are not waved through by lowering the gate — they get a justified inline ignore at the source (/* v8 ignore ... -- <reason> */ for TS; # pragma: no cover # <reason> for Python)
  • Coverage is a separate, deliberate command (npm run coverage), not baked into the fast validate loop — mirroring inspector's split (validate = fast unit tests; coverage = instrumented gate)

Scope

TypeScript servers (vitest + @vitest/coverage-v8 already in place):

Python servers (pytest + pyright + ruff in place; coverage tooling TBD):

  • fetch, git, time
  • Add pytest-cov/coverage.py; note coverage.py's fail_under is global, not per-file — pick the per-file enforcement mechanism as part of this work (e.g. coverage json + a small check script, matching the TS gate's semantics)
  • uv run coverage equivalent documented per server

Tasks

  • Audit current coverage per server — record the baseline in a comment on this issue (some servers may be far from 90; the audit sizes the work)
  • Write the missing tests, server by server (each server can be its own PR — independent, reviewable chunks)
  • Annotate genuinely-unreachable branches with justified ignores instead of gate-lowering
  • Enforce: per-file 90/90/90/90 thresholds in each TS vitest config; per-file mechanism for Python
  • Root coverage script (TS chain) + per-server Python invocation
  • Decide CI posture and document it: inspector runs validate in CI and keeps the coverage gate local-only — adopt the same unless there's a reason not to
  • Reintroduce the rule into AGENTS.md ("Always test new or modified code" section): per-file ≥ 90 on all four dimensions, justified-ignore policy, coverage vs validate split — wording adapted from the inspector reference

Dependencies

Acceptance

  • Every TS server passes its per-file 90% gate via npm run coverage
  • Every Python server passes the equivalent gate
  • Gates are enforced by configuration, not convention
  • AGENTS.md states the rule, and the rule is true on the day it's stated

Activity

  1. self-assigned this
    on Jul 4, 2026
  2. Math1987 commented on Aug 6, 2026

    @Math1987

    I audited the current filesystem baseline on main at 76d64c82 using Node 22.23.2.

    Command:

    cd src/filesystem
    npm ci
    npm test

    Results: 7 test files and 152 tests pass.

    File Statements Branches Functions Lines
    All files 45.91% 51.80% 40.32% 45.33%
    index.ts 0% 0% 0% 0%
    lib.ts 77.34% 65.75% 89.47% 76.92%
    path-utils.ts 94.44% 94.28% 100% 94.44%
    path-validation.ts 76.66% 80% 100% 76.66%
    roots-utils.ts 91.66% 71.42% 100% 91.66%

    The main structural issue is that the integration tests exercise the compiled server through child processes, so Vitest's V8 report records index.ts as completely uncovered even though those tests call many registered tools.

    I'd be happy to take the filesystem slice. Before starting the implementation, could you confirm the preferred direction for index.ts coverage?

    1. Refactor server construction/registration so the server can be exercised in-process by Vitest, or
    2. Collect and merge V8 coverage from the spawned server processes.

    Also, since #4473 is still open and main does not yet contain the planned AGENTS.md / validate setup, should this slice wait for #4473 or proceed independently and rebase afterward?

  3. Math1987 commented on Aug 14, 2026

    @Math1987

    Hi @cliffhall,

    I’m following up on the filesystem coverage slice. Before I start, would you prefer in process server construction for index.ts coverage, or collecting coverage from spawned processes? Should this work wait for #4473, or is an independent PR welcome?

  4. added this to the v2.0.0 milestone on Sep 26, 2026
  5. cliffhall commented on Sep 26, 2026

    @cliffhall
    MemberAuthor

    Thanks for the baseline audit, and sorry for the slow reply. This work is now part of the 2026-07-28 Spec Refactor tracker, #4857, and this issue is closing in favor of:

    Both come first in the tracker. They pin down current behavior as a regression net before the SDK v2 migrations (Parts 3 and 4) and the move to the 2026-07-28 spec (Parts 5 and 6).

    Answers to your two questions:

    1. Coverage of index.ts: go with option 1, building the server in the same process. Refactor server construction (e.g. a createServer() factory that index.ts calls) and exercise it with an SDK Client over an in-memory transport. Keep at most a thin spawn-based smoke test for the binary entry point. Tests should use only the public client API and wire-level results, so the Part 3 codemod can update them without changing any assertions.
    2. Adopt AGENTS.md (with CLAUDE.md pointer) modeled on the inspector repo #4473: no need to wait. The test and coverage work doesn't depend on it. Putting the rule back into AGENTS.md is tracked in Bugs, Coverage, and SDK Wave 1 Part 1: TypeScript servers: extensive unit tests + per-file 90% coverage gate #4854 as a follow-up once both suites land.

    You're very welcome to take the filesystem slice of #4854 as its own PR. Please target the v2/main branch (not main) and reference #4854 in the PR.

  6. removed this from the v2.0.0 milestone on Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions