Reduce CLI cold-start time with lazy imports and add an import-cost budget test

[dgenio]

Summary

Profile contextweaver --help and common subcommand startup, defer heavy imports (tiktoken encoding load, mcp, jsonschema, rich rendering machinery, benchmark-only modules) to first use, and lock the result with a CI import-cost budget test (python -X importtime derived).

Why this matters

The CLI is a primary touchpoint (demo-driven adoption, recipes, scripting), and the gateway is launched via contextweaver mcp serve by MCP clients at session start. Import-time loading of tokenizers/transport stacks taxes every invocation — including --help — and import-time regressions are invisible without a budget test. Snappy CLIs measurably improve first-run impressions.

Current evidence

• pyproject.toml core deps include tiktoken, mcp, jsonschema, typer, rich — all importable from contextweaver.__init__/__main__ paths; src/contextweaver/__init__.py re-exports broadly (296 lines), so import contextweaver pulls wide.
• tokens.py constructs TiktokenEstimator lazily via lru_cache (good precedent), but module-level imports elsewhere may still load tiktoken eagerly — measurement needed (this issue starts with profiling, not assumptions).
• No import-time test in tests/ (grep importtime: none).

External context

python -X importtime profiling plus deferred imports is the standard CLI-latency playbook; Typer supports lazy subcommand patterns.

Proposed implementation

1. Measure: commit a small scripts/ profile helper output to the PR (not the repo) showing the top import costs for --help, demo, route.
2. Apply targeted deferrals: function-local imports for mcp/jsonschema in _mcp_cli.py paths; ensure tiktoken encoding loads only on first count; audit __init__.py re-exports for heavy transitive pulls (consider module __getattr__ lazy re-exports — also useful for the alias-deprecation candidate).
3. Add a budget test: subprocess python -c "import contextweaver" and contextweaver --help wall-time/importtime ceiling with generous margin (regression tripwire, not a microbenchmark).

AI-agent execution notes

• Inspect first: __init__.py, __main__.py, _mcp_cli.py, tokens.py, protocols.py (TiktokenEstimator import site).
• Hard rule: no business logic in __init__.py — __getattr__-based lazy re-export is allowed only as re-export plumbing; confirm with maintainer if ambiguous (flag in PR).
• Don't break from contextweaver import X paths (API-map/compat tests must pass); mypy strict with lazy patterns needs care (TYPE_CHECKING imports).

Acceptance criteria

• Measured before/after numbers in the PR for import contextweaver and --help (target: meaningful reduction, stated honestly).
• Budget test in CI with documented threshold and margin rationale.
• No public import path removed (compat test green).

Test plan

Import-cost budget test; full suite for laziness-induced breakage (especially optional-extra guarded imports); make ci on all matrix cells.

Documentation plan

CHANGELOG; CONTRIBUTING note on keeping imports lazy; no user-docs change.

Migration and compatibility notes

Not expected to require migration.

Risks and tradeoffs

Lazy imports move failures from import time to call time (acceptable; error messages must stay clear); budget tests can flake on slow runners — use importtime metrics or generous wall-time margins.

Suggested labels

performance, developer-experience