Skip to content

The CLI was paying for a web server on every command (#173) - #777

Merged
Masterplanner25 merged 1 commit into
mainfrom
perf/173-cli-import-cost
Sep 6, 2026
Merged

Masterplanner25 merged 1 commit into
mainfrom
perf/173-cli-import-cost

Conversation

@Masterplanner25

Copy link
Copy Markdown
Owner

You were right that the CLI import cost had been looked at before — it was
measured during the #702 PyPy work and recorded as the blocker. What had not
been done is asking whether it was reducible. It is, and by a lot.

The finding

nodus.cli.cli imported nodus.services.server at module scope for the four
commands that need it (serve, snapshot, snapshots, restore). That module
imports FastAPI, uvicorn and pydantic — so nodus run, nodus fmt,
nodus check and nodus --version all paid for a web server nobody asked for.

Alternating before/after in one window, best of three each, so machine drift
shows as noise rather than signal:

before after
import nodus.cli.cli 1435 ms 652 ms 2.20x
nodus --version 1592 ms 640 ms 2.49x
nodus run hello.nd 1711 ms 1011 ms 1.69x

Roughly 700 ms off every CLI invocation.

Same shape as CIRC-001 (#103), which made vm.py's workflow-runner imports
lazy, and carrying the same instruction: do not hoist these back.

Why it is filed under #173

This is the standing blocker on that issue's PyPy path. PyPy's interpreter
starts faster than CPython's (104 ms vs 184 ms); what makes it unadoptable is
importing Nodus's own module tree — run-once code the JIT never warms, against
nodus run's 200 ms default deadline. This does not make PyPy adoptable on its
own, but it removes the largest single term.

The throughput ceiling is unchanged. This is startup, not dispatch.

Two things found on the way

#173's recorded throughput figure is stale. Measured with the VM's own
instruction counter rather than an estimate per iteration: 650–850K instr/s
at 17 instructions per iteration, against the ~200K the issue records.
tools/benchmark_runtime.py now reports startup and throughput so neither has
to be transcribed again. My own first reading assumed "about 6 instructions per
iteration" and was wrong by nearly 3x — which is why it uses the counter.

An unanchored assertion was reporting a defect that was not there.
assertNotIn("import re", CLI_SOURCE) also matches import restore_snapshot,
and did the moment these imports moved. It matches whole lines now, falsified
against both import re and from re import ....

Testing

tests/test_cli_import_cost.py asserts on sys.modules in a subprocess — a
behavioural test cannot see this (the CLI worked either way, only slower), and
by the time any other test runs something has usually imported FastAPI already.

Falsified by restoring the eager import: three subtests red, both controls
green. The controls matter here — an assertion that the CLI imports no web
framework is equally satisfied by a cli module that fails to import at all, so
one control checks it still imports what it uses and another checks the server
module still has its framework.

  • ruff clean · nodus_gate --static --closed-issues --shapes PASS (7/7, 0 new)
  • 418 passed across CLI/server/snapshot; remaining failures are
    test_cli_completion.py::BashExecutionTests, which shell out to bash, fail
    identically on this box before this branch, and are green on CI

🤖 Generated with Claude Code

https://claude.ai/code/session_01K4Y5EtwACvnLXh6MEtY82v

`nodus.cli.cli` imported `nodus.services.server` at module scope for the
four commands that need it -- `serve`, `snapshot`, `snapshots`, `restore`.
That module imports FastAPI, uvicorn and pydantic, so `nodus run`,
`nodus fmt`, `nodus check` and `nodus --version` all paid for a web server
nobody asked for. The imports are lazy now, inside those four commands.

Measured alternating before/after in one window, best of three each, so
machine drift shows as noise rather than signal:

  import nodus.cli.cli    1435ms -> 652ms   (2.20x)
  nodus --version         1592ms -> 640ms   (2.49x)
  nodus run hello.nd      1711ms -> 1011ms  (1.69x)

Roughly 700ms off every CLI invocation.

This is the standing blocker on #173's PyPy path, which is why it is filed
there. PyPy's *interpreter* starts faster than CPython's (104ms vs 184ms);
what makes it unadoptable is importing Nodus's own module tree -- run-once
code the JIT never warms, against `nodus run`'s 200ms default deadline.

Same shape as CIRC-001 (#103), which made vm.py's workflow-runner imports
lazy, and carrying the same instruction: do not hoist these back.

`tests/test_cli_import_cost.py` asserts on `sys.modules` in a subprocess.
A behavioural test cannot see this -- the CLI worked either way, only
slower -- and by the time any other test runs, something has usually
imported FastAPI already. Falsified by restoring the eager import: three
subtests red, both controls still green.

Two things found on the way:

**The recorded throughput figure was stale.** Measured with the VM's own
instruction counter rather than an estimate per loop iteration: 650-850K
instr/s at 17 instructions per iteration, against the ~200K #173 records.
`tools/benchmark_runtime.py` reports startup and throughput so neither
number has to be transcribed again. An earlier reading of mine assumed
"about 6 instructions per iteration" and was wrong by nearly 3x, which is
why the counter is used.

**An unanchored assertion was reporting a defect that was not there.**
`assertNotIn("import re", CLI_SOURCE)` also matches `import
restore_snapshot`, and did the moment these imports moved. It matches whole
lines now, and was falsified against both `import re` and
`from re import ...`.

The throughput ceiling itself is unchanged; this is startup, not dispatch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K4Y5EtwACvnLXh6MEtY82v
@Masterplanner25
Masterplanner25 merged commit 86f1cbf into main Sep 6, 2026
4 checks passed
@Masterplanner25
Masterplanner25 deleted the perf/173-cli-import-cost branch September 6, 2026 04:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant