Skip to content

Bugs, Coverage, and SDK Wave 1 Part 1: TypeScript servers: extensive unit tests + per-file 90% coverage gate #4854

Description

@cliffhall

Part of the 2026-07-28 Spec Refactor tracker #4857. This is the first step, and has no dependencies.

Goal

Before any SDK or spec changes, build an extensive vitest suite that pins down the current behavior of every TypeScript server (everything, filesystem, memory, sequentialthinking) on @modelcontextprotocol/sdk@1.x. The suite must pass the inspector's coverage quality gate from #4474.

This suite is the regression net for Part 3 (#4856). The SDK v2 migration is supposed to be transparent on the wire, and these tests are how we prove it. Existing tests may be deleted where the new suite covers them better.

Test design

  • Test through the protocol, in-process.
    • Connect an SDK Client to each server over an in-memory transport, and assert on what goes over the wire: tool, resource and prompt lists, call results, errors and notifications.
    • Refactor each server so it can be built in-process, e.g. with a createServer() factory that index.ts calls. Today the filesystem integration tests spawn the compiled server as a child process, so V8 reports index.ts as 0% covered (see the baseline in Bring all servers to the 90% per-file coverage gate; reintroduce the rule to AGENTS.md #4474). The in-process approach fixes that.
    • Keep at most a thin spawn-based smoke test for the binary entry point.
  • Stay migration-proof. Tests should touch only the public client API and wire-level shapes, not SDK internals or handler-context objects. In Part 3, the codemod should be able to update these tests with import and type changes alone. Any assertion that has to change during Part 3 is a behavior change and must be justified in that PR.
  • Coverage by server:
    • everything: every tool, resource (including templates), prompt and completion. Also:
      • logging simulation
      • subscriptions and list-changed notifications
      • roots, sampling and elicitation, driven by a test client that answers the server's requests
      • the experimental tasks tools
      • the stdio, SSE and Streamable HTTP transports
    • filesystem:
      • every tool
      • path validation, including symlink, traversal and unicode edge cases
      • allowed-directory and Roots handling
      • structured content and tool annotations
    • memory: knowledge-graph CRUD, persistence (atomic save, file path), search, and resource behavior.
    • sequentialthinking: input schema, and branching and revision logic.

Quality gate (from #4474, modeled on the inspector's v2/main AGENTS.md)

  • Threshold: ≥ 90 per file on all four dimensions (lines, statements, functions, branches). Set it in each server's vitest config: coverage: { provider: 'v8', thresholds: { perFile: true, lines: 90, statements: 90, functions: 90, branches: 90 } }.
  • Unreachable branches: don't lower the gate for code that genuinely can't be reached. Mark it with a justified inline ignore at the source instead: /* v8 ignore next -- <reason> */.
  • Separate coverage command: coverage is its own deliberate command (npm run coverage per server, and a root npm run coverage that chains the workspaces). If TypeScript workspace gate: Prettier, ESLint, root validate, CI #4864 lands first, it has already split each workspace's test from coverage, so this issue adds the thresholds to the existing coverage script. It is not part of the fast test / validate loop.
  • CI posture (amended): CI enforces the per-file coverage gate, in a parallel coverage job next to the fast tests, as the Inspector's CI now does (inspector#2159). Maintainers decided this on PR docs: agentic software factory inception (docs/agent-guidance-inception.md) #4861 (the agentic software factory inception doc). The coverage command stays separate from the fast test/validate loop locally. This issue wires it in (changed from the earlier plan, which left the wiring to Add local:gate, pre-push-gate skill #4871): add the parallel coverage job to typescript.yml and a TS coverage stage to local:gate, and record the stage in docs/quality-gate.md and the pre-push-gate skill. Add local:gate, pre-push-gate skill #4871 builds local:gate first, without coverage, so that it does not have to wait for this suite.

Acceptance criteria

  • Every TS server passes npm run coverage at 90/90/90/90 per file, enforced by config.
  • A PR that drops any TS file below 90 fails CI, and npm run local:gate fails on it too.
  • Every ignore comment carries a reason.
  • The suite passes on current main (SDK v1) in CI.
  • Removed tests are listed in the PR with the reason. Each server can be its own PR.

Carried over from #4474, which this issue supersedes

Activity

  1. added this to the v2.0.0 milestone on Sep 26, 2026
  2. self-assigned this
    on Sep 26, 2026
  3. changed the title [-]2026-07-28 Spec Refactor Part 4: Modern-era Python servers[/-] [+]2026-07-28 Spec Refactor Part 5: Extensive new-spec unit tests for TypeScript servers[/+] on Sep 26, 2026
  4. changed the title [-]2026-07-28 Spec Refactor Part 5: Extensive new-spec unit tests for TypeScript servers[/-] [+]2026-07-28 Spec Refactor Part 1: TypeScript servers: extensive unit tests + per-file 90% coverage gate[/+] on Sep 26, 2026
  5. cliffhall commented on Sep 28, 2026

    @cliffhall
    MemberAuthor

    Amended: the CI posture is now "CI enforces the per-file coverage gate in a parallel job", replacing "coverage stays local". Maintainers decided this on PR #4861, matching the Inspector's CI (inspector#2159). Wiring it into CI and local:gate is #4871; this issue still provides the coverage command.

  6. olaservo commented on Sep 29, 2026

    @olaservo
    Member

    Here is a coverage baseline for the four TypeScript servers to size this work, from main at f46d957 on 2026-09-28. 17 of 47 files meet the 90% gate. Helpers and pure functions are well tested. The code that registers and serves the tools is not, and that is the layer Part 3 changes. A few small refactors have to land before in-process tests can start. Issues that turned out not to be gaps are listed at the end.

    Baseline

    Server Tests passing Files at 90% Lowest files (lines %)
    everything 109 16 of 37 index.ts and all three transports 0, server/roots.ts 11.8, tools/trigger-elicitation-request-async.ts 19.6, prompts/completions.ts 25.0
    filesystem 168 1 of 5 index.ts 0, path-validation.ts 83.3, lib.ts 85.4
    memory 87 0 of 2 index.ts 83.5
    sequentialthinking 26 0 of 3 index.ts 0
    Per-file tables (Node 24.19, vitest 4.1.8, vitest run --coverage per package, same method as #4474)

    Bold is below the gate.

    everything

    File Lines Stmts Funcs Branches
    index.ts 0 0 0 0
    transports/stdio.ts, sse.ts, streamableHttp.ts 0 0 0 0
    server/roots.ts 11.8 11.8 0 0
    server/index.ts 60.0 54.5 25.0 0
    prompts/completions.ts 25.0 20.0 22.2 0
    tools/trigger-elicitation-request-async.ts 19.6 18.9 33.3 9.3
    tools/trigger-sampling-request-async.ts 29.7 28.9 33.3 16.7
    tools/get-roots-list.ts 53.8 53.8 33.3 23.1
    resources/files.ts 65.5 59.4 50.0 30.0
    resources/templates.ts 70.0 68.3 50.0 26.7
    resources/subscriptions.ts 79.5 77.5 75.0 50.0
    tools/simulate-research-query.ts 76.5 74.5 70.0 68.6
    tools/gzip-file-as-resource.ts 81.2 81.4 75.0 61.1
    resources/session.ts 85.7 85.7 100 75.0
    tools/trigger-elicitation-request.ts 90.9 84.2 100 63.3
    server/logging.ts 93.3 93.3 75.0 66.7
    tools/trigger-sampling-request.ts 100 100 100 75.0
    tools/get-annotated-message.ts, get-resource-links.ts 100 100 100 87.5
    the other 16 files 100 100 100 100

    filesystem

    File Lines Stmts Funcs Branches
    index.ts 0 0 0 0
    lib.ts 85.4 85.9 92.0 73.0
    path-validation.ts 83.3 83.3 100 90.0
    roots-utils.ts 91.7 91.7 100 71.4
    path-utils.ts 94.4 94.4 100 94.3

    memory

    File Lines Stmts Funcs Branches
    index.ts 83.5 84.3 84.2 79.0
    version.ts 90.0 90.0 100 50.0

    sequentialthinking

    File Lines Stmts Funcs Branches
    index.ts 0 0 0 0
    lib.ts 96.7 96.7 100 88.9
    version.ts 90.0 90.0 100 50.0

    Prerequisites

    • Every index.ts needs a createServer(...) export and a guarded main().
    • filesystem's allow-list (setAllowedDirectories in lib.ts) has to move from a module global to per-instance.
    • everything tests need to reset its module-level session maps or set serverTransport.sessionId before connect.
    • everything's sse.ts and streamableHttp.ts need a createApp() refactor before any HTTP test.
    • registrations.test.ts fails on a cold start. Hoisting its dynamic imports removes the cause.
    Why each one is needed
    • Every index.ts builds the server and connects stdio at module load. Importing one from vitest attaches the server to the runner's stdio.
    • filesystem also does a top-level await over process.argv. memory assigns its manager only inside main().
    • Two in-process filesystem servers cannot coexist while the allow-list is a module global.
    • The everything maps are in server/roots.ts, server/logging.ts, resources/subscriptions.ts, resources/session.ts and both toggle tools. InMemoryTransport has no sessionId, so every in-memory session shares the undefined key.
    • sse.ts and streamableHttp.ts call app.listen at import and export nothing.
    • registrations.test.ts does its dynamic imports inside the 5s default timeout. A cold start took 7s here.

    What the new suites should pin

    Each item was checked against the source on main.

    filesystem

    filesystem details

    memory

    memory details

    sequentialthinking

    everything

    everything details

    Two small inconsistencies block the branch gate:

    • blobResource returns mimeType: "text/plain" (templates.ts:108) against the template's application/octet-stream. That leaves get-resource-links.ts:76 dead.
    • trigger-elicitation-request.ts 198 and 207 are dead relative to the requested schema.

    Not gaps after all

    #499, #734 and #475 look closeable.

    The scan is scripted and can be rerun against v2/main as slices land, with per-file deltas against this baseline.


    Scan and verification done with help from Claude Code. The coverage numbers come from a real run; the per-issue verdicts were checked against the source but are a starting point, not a review.

  7. 6 remaining items

  8. cliffhall commented on Oct 4, 2026

    @cliffhall
    MemberAuthor

    Done in the stack #4970, #4973, #4977, #4978 and #4969, merged into v2/main. All four TypeScript servers now have in-process suites and a per-file 90% coverage gate, enforced in CI and in local:gate. Tests that pin a known bug carry a KNOWN BUG #N marker, and those bugs are tracked in #5004.

  9. changed the title [-]2026-07-28 Spec Refactor Part 1: TypeScript servers: extensive unit tests + per-file 90% coverage gate[/-] [+]2026-07-28 Spec Refactor Wave 1 Part 1: TypeScript servers: extensive unit tests + per-file 90% coverage gate[/+] on Oct 5, 2026
  10. changed the title [-]2026-07-28 Spec Refactor Wave 1 Part 1: TypeScript servers: extensive unit tests + per-file 90% coverage gate[/-] [+]Bugs, Coverage, and SDK Wave 1 Part 1: TypeScript servers: extensive unit tests + per-file 90% coverage gate[/+] on Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions