cli: exit-code map + structured JSON error envelopes #1564

Activity

1. cliffhall

   added

   v2Issues and PRs for v2Issues and PRs for v2

   on Jul 1, 2026
2. cliffhall

   added a parent issue on Jul 1, 2026

   • Tracking: re-implement PR #1510 (spec-conformant Apps host + programmatic review path) as scoped issues #1579
3. cliffhall

   self-assigned this

   on Jul 1, 2026
4. cliffhall

   linked a pull request that will close this issuefeat(cli): exit-code map + structured JSON error envelopes #1583on Jul 1, 2026
5. cliffhall commented on Jul 1, 2026

   

   cliffhall

   on Jul 1, 2026

   MemberAuthor

   More actions

   Smoke test & AGENTS.md audit — PR #1583

   Verified on branch v2/1564-cli-exit-codes (HEAD fa5514a3) against base v2/main.

   Smoke test

   Method: Built the CLI (clients/cli/build/index.js) and the bundled stdio test server, then drove the real built binary end-to-end with a small Node script (no mocks). Cases: a success path over stdio, two usage errors, an unreachable host (bad DNS over HTTP), a tool-call to a nonexistent tool, and an auth path against a local HTTP server that answers 401 with WWW-Authenticate. For each case the script captures the process exit code and the single JSON error envelope line on stderr.

   ### SUCCESS: tools/list (stdio test server)
   exit code : 0
   envelope  : (no JSON envelope on stderr)
   stdout[0] : {   (tools JSON printed)
   
   ### USAGE ERROR: missing --method
   exit code : 1
   envelope  : {"error":{"code":"error","message":"Method is required. Use --method to specify the method to invoke."}}
   
   ### USAGE ERROR: unsupported method foo/bar
   exit code : 1
   envelope  : {"error":{"code":"error","message":"Unsupported method: foo/bar. Supported methods include: tools/list, tools/call, resources/list, resources/read, resources/templates/list, prompts/list, prompts/get, logging/setLevel"}}
   
   ### UNREACHABLE: bad DNS host (http)
   exit code : 4
   envelope  : {"error":{"code":"unreachable","message":"fetch failed","cause":"getaddrinfo ENOTFOUND does-not-exist.invalid"}}
   
   ### TOOL: nonexistent tool name
   exit code : 0
   envelope  : (no JSON envelope on stderr)
   note      : server returns isError:true content on stdout (exit 0)
   
   ### AUTH REQUIRED: server returns 401
   exit code : 3
   envelope  : {"error":{"code":"auth_required","message":"Streamable HTTP error: Error POSTing to endpoint: {\"error\":\"invalid_token\"}","status":401}}
   

   Result: Exit codes 0 (OK), 1 (USAGE), 4 (UNREACHABLE), and 3 (AUTH_REQUIRED) all fire correctly, each with a well-formed single-line {"error":{…}} envelope — including the enriched cause (undici ENOTFOUND) on the unreachable case and status: 401 on the auth case. Classification works as designed.

   Scope note (not a defect): EXIT_CODES.TOOL_ERROR (5) and EXIT_CODES.NO_APP (2) are defined in the map and covered by codeForExit/classifyError, but there is no CliExitCodeError throw site in cli.ts yet for them. A tools/call to an unknown tool returns an isError:true result on stdout and exits 0 (see the TOOL case above). This PR lands the exit-code map + envelope infrastructure; wiring codes 2 and 5 to their throw sites is follow-up work.

   AGENTS.md compliance

   • Mantine idioms: N/A — this is the CLI, no React/Mantine surface.

   • TypeScript: PASS. git diff origin/v2/main...HEAD adds no any (no : any, as any, <any>), and no eslint-disable / @ts-ignore / @ts-expect-error / rule-suppression comments. Error shapes are narrowed via typed guards (statusOf, causeOf).

   • Coverage gate (cd clients/cli && npm run test:coverage): PASS. 129 tests pass. src/error-handler.ts:

     Dimension	%	≥90
     Statements	100	✅
     Branches	94.87	✅
     Functions	100	✅
     Lines	100	✅

     (src/cli.ts also clears the gate: 100 / 91.13 / 100 / 100.)

   Verdict: PASS

   Exit-code map and JSON error envelopes behave correctly end-to-end on the real binary; TypeScript and the per-file coverage gate both pass. The only observation is a scope note (codes 2/5 defined but not yet emitted), which is consistent with this PR being the infrastructure layer.

   Custom smoke-test script

   #!/usr/bin/env node
   /**
    * Smoke test for PR #1583 — CLI exit-code map + structured JSON error envelopes.
    * Drives the BUILT cli binary (clients/cli/build/index.js) against the bundled
    * stdio test server (success + tool paths), a bad DNS host (unreachable), and a
    * local HTTP server that answers 401 (auth). Prints, for each case:
    *   case -> exit code -> single JSON stderr envelope line
    */
   import { spawn } from "node:child_process";
   import { mkdtempSync, writeFileSync } from "node:fs";
   import { tmpdir } from "node:os";
   import { join } from "node:path";
   import http from "node:http";
   
   const repoRoot = process.argv[2];
   const cli = join(repoRoot, "clients", "cli", "build", "index.js");
   const stdioServer = join(repoRoot, "test-servers", "build", "test-server-stdio.js");
   
   const work = mkdtempSync(join(tmpdir(), "smoke-exit-"));
   const catalogPath = join(work, "catalog.json");
   writeFileSync(
     catalogPath,
     JSON.stringify({
       mcpServers: {
         test: { type: "stdio", command: process.execPath, args: [stdioServer] },
       },
     }),
   );
   
   function run(args) {
     return new Promise((res) => {
       const child = spawn(process.execPath, [cli, ...args], {
         cwd: repoRoot,
         env: { ...process.env, HOME: work, USERPROFILE: work },
       });
       let stdout = "", stderr = "";
       child.stdout.setEncoding("utf-8");
       child.stderr.setEncoding("utf-8");
       child.stdout.on("data", (d) => (stdout += d));
       child.stderr.on("data", (d) => (stderr += d));
       child.on("close", (status) => res({ status, stdout, stderr }));
     });
   }
   
   function envelopeLine(stderr) {
     const lines = stderr.trim().split("\n").filter(Boolean);
     for (let i = lines.length - 1; i >= 0; i--) {
       const l = lines[i].trim();
       if (l.startsWith("{") && l.includes('"error"')) return l;
     }
     return "(no JSON envelope on stderr)";
   }
   
   function report(name, r, note) {
     console.log(`\n### ${name}`);
     console.log(`exit code : ${r.status}`);
     console.log(`envelope  : ${envelopeLine(r.stderr)}`);
     if (note) console.log(`note      : ${note}`);
     const firstStdout = r.stdout.trim().split("\n")[0] ?? "";
     if (firstStdout) console.log(`stdout[0] : ${firstStdout.slice(0, 120)}`);
   }
   
   function start401() {
     return new Promise((res) => {
       const srv = http.createServer((req, resp) => {
         resp.writeHead(401, {
           "WWW-Authenticate": 'Bearer realm="mcp", error="invalid_token"',
           "Content-Type": "application/json",
         });
         resp.end(JSON.stringify({ error: "invalid_token" }));
       });
       srv.listen(0, "127.0.0.1", () => res(srv));
     });
   }
   
   const base = ["--catalog", catalogPath, "--server", "test"];
   
   report("SUCCESS: tools/list (stdio test server)", await run([...base, "--method", "tools/list"]));
   report("USAGE ERROR: missing --method", await run([...base]));
   report("USAGE ERROR: unsupported method foo/bar", await run([...base, "--method", "foo/bar"]));
   report("UNREACHABLE: bad DNS host (http)", await run([
     "--transport", "http", "--server-url", "http://does-not-exist.invalid/mcp", "--method", "tools/list",
   ]));
   report("TOOL: nonexistent tool name", await run([
     ...base, "--method", "tools/call", "--tool-name", "no_such_tool",
   ]));
   const srv = await start401();
   const port = srv.address().port;
   report("AUTH REQUIRED: server returns 401", await run([
     "--transport", "http", "--server-url", `http://127.0.0.1:${port}/mcp`, "--method", "tools/list",
   ]));
   srv.close();
6. cliffhall commented on Jul 2, 2026

   

   cliffhall

   on Jul 2, 2026

   MemberAuthor

   More actions

   🔗 Implemented by PR #1583. Its code is already integrated into the Wave 1 rollup branch 1579-wave-1 (see rollup PR #1601 / verification #1600). #1583 currently targets v2/main and will auto-show as Merged once the rollup lands in v2/main. The Development panel doesn't link it now because the closing keyword is on a non-default base branch.
7. cliffhall

   closed this as completedon Jul 9, 2026
8. pull

   added a commit that references this issue on Jul 28, 2026

   feat(cli): exit-code map + structured JSON error envelopes (closes mo…

   ...

   This issue will close once commit a5d8d80 is merged into the 'main' branch.

   a5d8d80
9. cliffhall

   added this to the v2.0.0 milestone on Jul 28, 2026