Wave 3: Error branch tests + production error handling fixes - #21
Conversation
ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Free Run ID: 📒 Files selected for processing (11)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (6)
📝 WalkthroughWalkthroughAdded granular error handling across CLI bootstrap and command flows: filesystem and Fernet key creation, database initialization, key protection/enrollment, proxy/process startup, and wrapping CLI commands now catch specific exceptions and convert them to WorthlessError codes; extensive tests and property tests exercise these failure paths and security invariants. Changes
Sequence Diagram(s)sequenceDiagram
participant CLI
participant FS as Filesystem
participant DB as Database (sqlite3)
participant Proc as Subprocess/Proxy
CLI->>FS: ensure .worthless dir + permissions
alt OSError / PermissionError
FS-->>CLI: raise OSError
CLI-->>CLI: wrap -> WorthlessError(ErrorCode.BOOTSTRAP_FAILED)
else success
CLI->>FS: ensure fernet.key exists (create if missing)
alt write error
FS-->>CLI: raise OSError
CLI-->>CLI: wrap -> WorthlessError(ErrorCode.BOOTSTRAP_FAILED)
else success
CLI->>DB: _init_db(worthless.db)
alt sqlite3.DatabaseError / OSError
DB-->>CLI: raise DatabaseError/OSError
CLI-->>CLI: wrap -> WorthlessError(ErrorCode.SHARD_STORAGE_FAILED)
else success
CLI-->>CLI: return home ready
end
end
end
sequenceDiagram
participant UserCLI as CLI Command
participant Repo as ShardRepository/DB
participant FS as Filesystem
participant Proc as Proxy/Subprocess
UserCLI->>Repo: retrieve/store operations
alt Repo raises sqlite3.DatabaseError
Repo-->>UserCLI: raise DatabaseError
UserCLI-->>UserCLI: wrap -> WorthlessError(ErrorCode.SHARD_STORAGE_FAILED)
else success
UserCLI->>FS: read/write shard files / .env
alt FS raises OSError/PermissionError
FS-->>UserCLI: raise OSError
UserCLI-->>UserCLI: wrap -> WorthlessError(ErrorCode.BOOTSTRAP_FAILED or UNKNOWN)
else success
UserCLI->>Proc: spawn proxy / child process
alt subprocess.Popen fails
Proc-->>UserCLI: raise OSError
UserCLI-->>UserCLI: print WorthlessError(ErrorCode.PROXY_UNREACHABLE) / exit(1)
else success
Proc-->>UserCLI: running
end
end
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~55 minutes Poem
Note 🎁 Summarized by CodeRabbit FreeYour organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login. Comment |
Add compensation path tests for lock (shard_a write failure, .env rewrite failure, symlink rejection) and multi-enrollment unlock tests (ambiguity error when env_path=None, partial unlock leaves other enrollment intact). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add wrap tests for spawn_proxy failure, health timeout, child spawn failure, and _cleanup_proxy edge cases. Add up tests for corrupt/missing PID files, check_pid, and write/read roundtrip. Move inline imports to top-level. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Hypothesis-powered tests for crypto invariants: split/reconstruct roundtrip, shard independence, zero-after-use, repr redaction, tamper detection (bit-flip in shard_a/shard_b/commitment), and edge cases (empty key, type errors, length mismatch). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Brutus/Karen review identified gaps and quality issues. This commit: - Add test_wrap_proxy_crash_mid_session_warns (_watch_proxy thread path) - Add test_up_daemon_mode_writes_pid (full daemon flow PID lifecycle) - Add TestGateBeforeDecrypt (SR-03): static + AST analysis proving rules_engine.evaluate precedes repo.decrypt_shard - Add TestSanitizeNeverLeaksMessage (SR-05): Hypothesis property tests proving upstream error messages never leak through sanitization - Add _KEY_BINARY + _KEY_ANY strategies for non-ASCII byte coverage - Add nonce tamper test (test_flipped_nonce_bit_detected) - Add .env integrity assertion to env_rewrite_failure compensation test - Fix repr redaction tests to check shard hex, not key text (false positive: Hypothesis key "SplitRes" matched class name) - Document zero-after-use limitations (CPython GC, immutable copies) - Rename TestShardIndependence → TestShardNonEquality (honest naming) - Document multi-enrollment test as internal API contract, not CLI test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Every CLI command now catches bare exceptions and wraps them in WorthlessError with clean WRTLS-NNN messages instead of raw Python tracebacks. Fixes: - bootstrap.py: ensure_home wraps OSError → WRTLS-100, DB init → WRTLS-103 - lock.py: compensation re-raise now wraps in WRTLS-103, outer catch-all - unlock.py: outer catch-all for unhandled exceptions - up.py: outer try/except wrapping entire command, daemon Popen failure - wrap.py: outer try/except wrapping entire command Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tests for all production error handling paths added in previous commit: - test_cli_bootstrap.py: new file — read-only home dir, DB init failure - test_cli_lock.py: .env integrity assertion, catch-all wrapping - test_cli_unlock.py: repo.retrieve failure → WRTLS, catch-all wrapping - test_cli_up.py: daemon Popen failure, foreground proxy crash mid-session, daemon PID lifecycle, catch-all wrapping - test_cli_wrap.py: proxy crash mid-session (_watch_proxy thread), catch-all wrapping Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ve 3) Three fixes from security + architecture review: - unlock.py: Move try/finally scope to cover shard_a from allocation through ambiguity check. Previously, shard_a was not zeroed if the multi-enrollment ambiguity error raised before the try block. - bootstrap.py: Narrow _init_db catch from bare Exception to (OSError, sqlite3.DatabaseError) so programming errors propagate instead of being masked as SHARD_STORAGE_FAILED. - test_cli_bootstrap.py: Tighten assertions to check specific WorthlessError codes instead of accepting either WorthlessError or the original exception type. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
f2be29c to
5873470
Compare
…iled" (M12) Semgrep's python-logger-credential-disclosure rule pattern-matches on the word "Token" in log message strings, suspecting auth-token leakage. The flagged log call (proxy/app.py inside _do_record_spend) was using "Token" to mean LLM usage-tokens (the count-of-tokens-consumed for metering), not an auth token — neither the alias substitution (a SHA-256 fingerprint, public identifier) nor the provider substitution (a wire- protocol enum) is a credential. Pure terminology overlap, true false positive. The previous # nosemgrep annotation was correct but fragile (rule re-emits on dashboard refresh; alerts pinned to old commits don't auto-clear). Renaming the message to "Usage extraction failed" matches the function it lives in (_do_record_spend, which does USAGE extraction via extract_usage_openai/extract_usage_anthropic), removes the lexical trigger, and lets us drop the suppression annotation entirely. No behavior change. The 139 proxy + hardening + metering tests pass unchanged (no tests pin the log message literal). Refs: PR #127 — closes the last remaining Semgrep OSS alert (#21, anchored to refs/heads/main commit e9b2e8f). Annotation drop is the honest fix; renaming a log message is cheaper than maintaining a nosemgrep that gets re-flagged on every dashboard re-scan.
…am) (#127) * feat(cli): bundled TOML provider registry with user override (8rqs Phase 1) The first plumbing piece for worthless-8rqs (lock multiple LLM providers side-by-side): a TOML registry mapping known upstream URLs to wire protocol, so `worthless lock` can auto-detect the protocol when scanning .env files. What ships: - src/worthless/providers.toml — bundled with 6 entries (openai, anthropic, openrouter, groq, together, ollama). Practical-set decision; users can add more locally without touching the package. - src/worthless/cli/providers.py — loader. Reads bundled file via importlib.resources, optionally merges ~/.worthless/providers.toml on top (user wins on URL conflict). Malformed user file logs a warning and falls back to bundled-only — never crashes lock. - pyproject.toml — adds tomli to runtime deps (3.10 fallback for tomllib); package-data line so providers.toml ships in the wheel; deptry DEP001 whitelist for stdlib tomllib. - uv.lock — regenerated for the new runtime tomli dep. TDD: 12 tests in tests/cli/test_providers_registry.py, all passing. Mutation-tested: dropping the openai entry from providers.toml fails test_bundled_registry_has_six_providers AND test_bundled_includes_openai with clear diagnostics. Phase 1 of the worthless-8rqs plan. * feat(cli): worthless providers list/register subcommands (8rqs Phase 2) Surface the registry to users: - `worthless providers list` prints the merged registry as a table (NAME, PROTOCOL, SOURCE, URL). Source is "bundled" or "user". - `worthless providers list --json` (via the global -j/--json flag) emits machine-readable JSON. - `worthless providers register --name --url --protocol [--force]` appends a custom provider to ~/.worthless/providers.toml. Atomic write (.tmp + rename), mode 0644 (registry is public data). Validation up front: - name: alphanumeric + hyphen/underscore (regex) - url: scheme in {http, https}, non-empty netloc — rejects `not-a-url`, `javascript:alert(1)`. Accepts `http://localhost:1234/v1` (Ollama). - protocol: must be in {openai, anthropic}. - name conflict with bundled → refused with hint. - url conflict with bundled → refused unless --force. Surface changes outside the new module: - `cli/providers.py`: rename `_load_bundled`/`_load_user` → public `load_bundled`/`load_user`; add `bundled_names()` and `user_registry_path()` helpers used by the register command. - `cli/errors.py`: add `ErrorCode.INVALID_INPUT = 112` for validation failures (no existing code matched). - `cli/app.py`: register the new subcommand group. TDD: 13 tests in tests/cli/test_providers_command.py, all passing. Phase 1 tests still green (12/12). Total Phase 1+2 coverage: 25 tests. Phase 2 of the worthless-8rqs plan. * feat(storage): add shards.base_url with VACUUM INTO backup, no backfill (8rqs Phase 3) The DB-side groundwork for per-enrollment routing. After this commit the schema has the column; readers will start using it in Phase 4. What ships: - src/worthless/storage/schema.py: * SCHEMA gains `base_url TEXT` on the `shards` table (nullable). * migrate_db gains a guarded ALTER for the column. * BEFORE the ALTER fires (and only the first time), the DB is snapshotted via SQLite's VACUUM INTO 'db.sqlite.bak.<timestamp>' — a clean WAL-aware copy that gives the user a rollback option if anything goes sideways on a 0-user PR landing. * NO backfill. Brutus's catch from plan time: backfilling NULL → per- provider default would silently mis-route an OpenAI-protocol enrollment that pointed at OpenRouter via the old WORTHLESS_UPSTREAM_OPENAI_URL workaround. Phase 4's reader will refuse to read a NULL row with a clear "predates upstream URL storage; run worthless relock <name>" error, forcing explicit re-lock. Tests (4 new in tests/test_storage.py): - test_fresh_db_has_base_url_column — init_db creates the column from CREATE - test_migrate_adds_base_url_column — pre-8rqs DB gets the column - test_migrate_base_url_idempotent — running migrate twice is safe - test_migrate_creates_backup_before_altering — backup file appears - test_migrate_no_backup_when_already_migrated — no spam on idle migrate All 28 storage tests still green. Mutation-tested: removing the VACUUM INTO line makes test_migrate_creates_backup_before_altering fail with "no pre-migration backup created in <tmp>; existing files: [...]". Phase 3 of the worthless-8rqs plan. * refactor(storage): per-enrollment routing in repo API (8rqs Phase 4) The data layer now carries base_url end-to-end. Phases 5-6 will start USING this; Phase 4 just plumbs it through. Surface changes: - EncryptedShard NamedTuple gains `base_url: str | None = None` (additive, default keeps every existing constructor working — verified across 10+ call sites in tests + cli/commands/unlock.py). - __repr__ updated to include base_url (alongside the existing length- redacted byte fields). - ShardRepository.store(..., base_url=None) — new kwarg, INSERT plumbed. - ShardRepository.store_enrolled(..., base_url=None) — new kwarg, INSERT plumbed (the path Phase 7's lock command will use). - ShardRepository.fetch_encrypted SELECT now includes base_url; the returned EncryptedShard.base_url is None for legacy rows. Renamed (no shim): list_aliases_with_provider → list_aliases_with_routing. - Returns 4-tuples (alias, var_name, base_url, protocol). - Now JOINs shards × enrollments so callers get var_name in the same query (used by Phase 8's wrap rewrite). - Single existing caller (cli/commands/wrap.py:_list_enrolled_aliases) updated in this commit; for now slices the 4-tuple back to (alias, protocol) so existing _build_child_env / _PROVIDER_ENV_MAP logic keeps working until Phase 8 deletes that path. Tests: - 3 existing list_aliases_with_provider tests renamed + adapted to the new 4-tuple shape. - 2 new tests pin the base_url roundtrip: test_store_enrolled_with_base_url_roundtrips, test_store_enrolled_without_base_url_returns_none. - 1 new test pins legacy null behaviour: test_list_aliases_with_routing_legacy_row_has_null_base_url. 31 storage tests pass (was 28, +3 net). 92 storage+cli tests pass with no regressions. 169 storage+cli+security+proxy_hardening tests pass — confirms the EncryptedShard field-add is non-breaking for all existing consumers. Phase 4 of the worthless-8rqs plan. * feat(routing): per-enrollment base_url end-to-end (8rqs Phase 5+6) Adapters and proxy now thread the upstream URL per request. The legacy WORTHLESS_UPSTREAM_OPENAI_URL / _ANTHROPIC_URL env vars are GONE — no back-compat (no users). Adapter signature change (Phase 5): - adapters/types.py: ProviderAdapter Protocol gains required base_url kwarg. - adapters/openai.py: drop module-level UPSTREAM_URL constant + os.environ read; prepare_request(*, ..., base_url) computes f"{base_url.rstrip('/')}/chat/completions". - adapters/anthropic.py: same shape; appends "/messages". Proxy plumbing (Phase 6): - proxy/app.py: after fetch_encrypted, if encrypted.base_url is None (legacy row predating 8rqs), return 503 with hint "alias <name> predates upstream URL storage; run 'worthless relock' to re-enroll with --base-url" rather than fall back to a possibly-wrong default. Otherwise pass base_url=encrypted.base_url to prepare_request. Test surface updates (mechanical, mostly bulk-applied): - tests/test_adapters.py: 13 prepare_request call sites updated + TestPerEnrollmentBaseURL class added (6 new tests pinning the contract). - tests/test_adapter_bytearray.py: 5 sites updated. - tests/test_properties.py: 4 sites updated. - tests/test_contract.py: replace UPSTREAM_URL monkey-patches (test pollution risk under xdist) with proper enrollment-time base_url pointing at the mock upstream. Removed `import _oai_mod, _anth_mod`. - tests/test_proxy*.py: 8 store() fixture sites updated with base_url=https://api.openai.com/v1 so per-enrollment routing kicks in. - tests/test_error_metering_and_hardening.py: 1 site. - SKILL.md: documents the new `worthless providers list/register` subcommand group (was an undocumented-commands-drift test failure). Mutation-tested for the env-rip: - Re-add `os.environ.get("WORTHLESS_UPSTREAM_OPENAI_URL", ...)` to adapters/openai.py and run TestPerEnrollmentBaseURL::test_env_var_has_no_effect_on_openai with the env var set to https://attacker.example/v1. - Test FAILS with "'attacker' is contained here:" — exactly the regression we want to catch. Mutation-tested for the proxy refusal of NULL base_url: covered by the test_contract.py update — when an enrollment has base_url=None the proxy would return 503 to the contract test, breaking it; the test passes only because we now enroll WITH base_url. Full suite: 1846 passed, 10 skipped, 0 failures. Phases 5+6 of the worthless-8rqs plan, combined into one commit because the adapter signature change forces the proxy call site update in the same atomic change. * feat(cli): lock reads per-enrollment base_url from .env (8rqs Phase 7) The user-facing piece. Multi-provider .env now flows end-to-end: 1. User has OPENROUTER_API_KEY + OPENROUTER_BASE_URL in .env 2. worthless lock reads BOTH, stores the URL in DB, rewrites both VALUES in place (var names sacred — OPENROUTER_BASE_URL stays OPENROUTER_BASE_URL, never replaced with OPENAI_BASE_URL) 3. SDK auto-config reads the rewritten .env, sends to local proxy at the per-alias URL, proxy reconstructs and forwards upstream Surface changes (cli/commands/lock.py): - _derive_base_url_var(var_name, provider): preserves user naming (OPENROUTER_API_KEY → OPENROUTER_BASE_URL); falls back to _PROVIDER_ENV_MAP when var_name doesn't match the conventional shape. - _resolve_upstream_base_url(base_url_var, env_values, provider): prefers user's explicit *_BASE_URL value; falls back to bundled registry default for the provider name. - _PlannedUpdate gains base_url_var: str = "" (carries the derived var name through pass-1 → batch_rewrite for atomic single-call rewrite). - _pass1_db_writes accepts env_values and threads base_url= into both store_enrolled call sites (re-enroll branch + fresh-enroll branch). - _batch_rewrite uses p.base_url_var per-key. When the var already exists in .env, OVERWRITE the value (upstream is captured in DB); when missing, ADD with stderr notice ("worthless: added OPENAI_BASE_URL=... to .env (was missing)"). - _enroll_single (no-.env path) falls back to registry default. Surface change (cli/providers.py): - lookup_by_name(name) walks the merged registry by name. Used by lock.py to resolve provider → URL when no user var is set. Tests: - tests/test_cli_lock.py adds test_lock_reads_existing_base_url_from_env: asserts (a) DB row stores https://openrouter.ai/api/v1, (b) .env retains OPENROUTER_BASE_URL (NOT OPENAI_BASE_URL), (c) value points at local proxy URL containing the alias. - 7 existing TestLockFormatPreserving tests still pass — the auto-create-when-missing path for OpenAI keys without BASE_URL is unchanged. Note: re-applied in dedicated worktree at worthless-8rqs/ after a concurrent session repeatedly switched the main checkout's branch mid-edit and discarded uncommitted Phase 7 work. Phase 7 of the worthless-8rqs plan. * refactor(cli): wrap delegates BASE_URL ownership to lock (8rqs Phase 8) Pre-8rqs, ``worthless wrap`` synthesised ``OPENAI_BASE_URL= http://127.0.0.1:PORT/<alias>/v1`` into the child process env, hardcoded to ``OPENAI_*`` / ``ANTHROPIC_*`` regardless of what env-var name the user actually used in their .env. After 8rqs Phase 7, ``worthless lock`` writes the right local-proxy URL into the user's own variable name (e.g. ``OPENROUTER_BASE_URL``, ``GROQ_BASE_URL``) at lock time. Wrap injecting on top of that is double-work that also breaks for non-canonical providers. Surface changes: - ``cli/commands/wrap.py``: deleted module-level ``_PROVIDER_ENV_MAP``; ``_build_child_env`` is now a passthrough (returns ``dict(os.environ)``); signature kept for test compatibility but ``port`` and ``aliases`` are ignored. Module docstring updated to reflect the new contract. - ``cli/commands/lock.py``: pulled the small ``_PROVIDER_ENV_MAP`` constant in-house (still needed for the fallback in ``_derive_base_url_var`` when var_name doesn't match ``*_API_KEY``). - ``cli/commands/unlock.py``: changed import from ``wrap`` to ``lock`` (the dict moved). No behaviour change in unlock. Tests: - ``tests/test_cli_wrap.py``: - Deleted obsolete tests asserting injection (``test_child_env_anthropic``, ``test_child_env_multiple_providers``, ``test_unknown_provider_skipped``, ``test_multi_alias_same_provider_*``). - Added ``test_child_env_no_injection``: with parent env clean, child env has NONE of OPENAI/ANTHROPIC/OPENROUTER_BASE_URL. - Added ``test_parent_baseurl_passes_through``: if parent has OPENROUTER_BASE_URL set (typical post-lock state), wrap forwards it unchanged. - ``tests/test_e2e.py::test_wrap_real_proxy_transit``: child script updated to read .env via WORTHLESS_E2E_ENV_PATH (mimics how SDKs use ``python-dotenv``). The post-8rqs flow IS lock writes .env → SDK reads .env, so the test now exercises the real production path. Result: 1844 passing, 10 skipped, 0 failures (full suite excl docker-e2e). Note: re-applied in dedicated worktree at worthless-8rqs/ after concurrent sessions repeatedly switched the main checkout's branch mid-edit. Phase 8 of the worthless-8rqs plan. * fix(scanner): lower entropy threshold 4.5 → 3.9 to admit OpenRouter keys (8rqs prereq) Real OpenRouter keys have entropy ~4.118 — under the historical 4.5 threshold, scan + lock filter them as 'placeholders' and refuse to enroll, breaking the entire 8rqs multi-provider story. Mirrors PR #116 commit f087180 which fixed this on the WOR-306 sidecar epic. That epic hasn't merged to main yet, so the branch this PR is built on (off main 582a64f) still has the buggy 4.5 threshold. Bumping inline so 8rqs can ship; once the epic merges to main, this line will be identical and the merge a no-op. The same placeholder values that were verified to fall below 3.9 in PR #116 stay rejected (sk-your-key-here: 3.03, sk-aaaa: 0.88, WRTLS-decoy: 3.63, sk-PLACEHOLDER: 3.74). * fix(security): hoist NULL base_url check above reconstruction (SR-03) Blocker M1 from PR #127 expert review. The 8rqs Phase 6 commit (eba949e) placed the 'if encrypted.base_url is None: refuse' guard at proxy/app.py line 382 — AFTER both rules_engine.evaluate (line 329) AND key reconstruction (lines 363/372). A legacy row missing base_url triggered full key materialisation in memory before being refused with 503. That violates SR-03 (gate before reconstruct). This commit moves the guard to right after fetch_encrypted at line ~313, BEFORE shard_a extraction, body read, rules engine, and reconstruction. The denial path no longer touches key material. The minimum 6-LOC inline move is intentional. worthless-2pdi (P1 follow-up bead, blocked-by:worthless-8rqs) will promote this guard into a structural validate_encrypted_row() helper covering ALL row-shape denials (NULL base_url, missing prefix/charset, unknown adapter, malformed protocol) — see seam 1 in worthless-8rqs design notes. The hoist here is forward-compatible with that helper. Test: tests/test_proxy_hardening.py::TestGateBeforeDecrypt:: test_null_base_url_refused_before_reconstruction. Mocks worthless.proxy.app.reconstruct_key + reconstruct_key_fp; injects a row with base_url=None via fetch_encrypted monkey-patch; sends a real request through the proxy; asserts response 503 AND mock_reconstruct.call_count == 0 AND mock_reconstruct_fp.call_count == 0. Mutation-tested: replacing the hoisted condition with `if False and encrypted.base_url is None` causes the test to fail (reconstruct_key_fp gets called downstream), proving the test pins the new ordering. 122 proxy tests pass. Refs: worthless-8rqs (this PR), worthless-2pdi (root closure follow-up) * fix(proxy): strip Content-Encoding/Content-Length on decompressed responses Blocker M2 from PR #127 expert review. The proxy reads upstream response bodies via httpx aread()/aiter_bytes(), which auto-decompresses gzip per the Content-Encoding header. The decompressed bytes were then forwarded to the SDK with the ORIGINAL Content-Encoding: gzip header still attached — so SDKs (which trust the header) tried to gunzip plain JSON and got 'Error -3 while decompressing data: incorrect header check'. The PR #127 live-smoke required setting Accept-Encoding: identity on every request to bypass — meaning default SDK calls were broken. This commit drops Content-Encoding and Content-Length from forwarded response headers in adapters/types.py::relay_response. The body is already decompressed by httpx; without those headers, SDKs treat the response as plain JSON and parse correctly. Test: tests/test_proxy_hardening.py::TestGateBeforeDecrypt:: test_proxy_response_pipe_is_consistent_with_gzip_upstream. respx mocks upstream returning gzipped JSON with Content-Encoding: gzip; client reads the proxy's response and asserts internal consistency (either header gone + decompressed body, or header present + still-gzipped body). Without the fix, httpx's auto-decoder raises DecodingError before the test can even read the body — same error class users hit. Mutation-tested: reverting the _filter_response_headers call to dict(response.headers) reproduces the exact DecodingError, proving the test pins the user-visible bug. This is the M2 minimum (~5 LOC). The P2 follow-up bead worthless-yo9o deepens to true byte-transparent pipe (aiter_raw + async metering reconciliation off the response path) — see seam 3 in worthless-8rqs design notes. 169 proxy + adapter + contract tests pass. Refs: worthless-8rqs (this PR), worthless-yo9o (deeper pipe follow-up), worthless-wl4p (SR-11 byte-transparency rule follow-up) * fix(cli): lock refuses unregistered upstream URLs (Blocker #1) Blocker M3 from PR #127 expert review. Pre-fix, lock pulled the user's *_BASE_URL value from .env into the DB unchanged. An attacker who can write to .env (in-scope per project threat model) could set OPENROUTER_BASE_URL=https://attacker.example/v1 and the proxy would forward reconstructed shard-A as Bearer to attacker.example — credential exfiltration channel directly contradicting the per-enrollment routing claim of PR #127. This commit adds a registry-membership check on the user-supplied URL in _resolve_upstream_base_url. If the URL is not in the merged registry (bundled providers.toml + ~/.worthless/providers.toml user override), lock refuses with WorthlessError(INVALID_INPUT) and a hint to register: 'worthless providers register --name <n> --url <url> --protocol ...' The DB transaction never starts on the refused path — no enrollment created, no shard split. Test: tests/test_cli_lock.py::TestLockFormatPreserving:: test_lock_refuses_unregistered_attacker_base_url. Sets OPENROUTER_BASE_URL= https://attacker.example/v1 in .env, asserts: (a) exit_code != 0, (b) error message names the rejected URL, (c) error message hints at 'worthless providers register', (d) DB has zero enrollments after refusal. Mutation-tested: replacing the registry-check condition with `if False` allows the attacker URL through and fails the test. This is M3 minimum (15 LOC + 1 test). The follow-up beads deepen the defense: worthless-rzi1 (P1) adds per-request DB-tamper re-validation (closes the post-lock variant); worthless-8fbg (P1) hardens _validate_url itself (rejects RFC1918/loopback unless registry-marked internal=true); worthless-99ox (P1, folded from P2) adds adapter allowed_hosts for defense-in-depth. See seam 2 in worthless-8rqs design notes. 72 cli/lock + providers tests pass. Refs: worthless-8rqs (this PR), worthless-rzi1 + worthless-8fbg + worthless-99ox (P1 follow-ups), worthless-k5tk (SR-10 semgrep enforcement) * fix(cli): warn on non-canonical API key var name (Blocker #2) Blocker M4 from PR #127 expert review (product-manager). User's .env may contain non-canonical API-key vars like MY_OPENAI_KEY that don't match the <PROVIDER>_API_KEY convention. Brutus surfaced a narrow proxy-bypass scenario: the app reads MY_OPENAI_KEY explicitly and constructs an SDK client without base_url= override; the SDK falls through to api.openai.com and shard-A leaks on the wire instead of hitting the proxy. Per product-manager review the right behavior is a SOFT warning (not a refusal). The warning names the var, points at the BASE_URL that lock will set, and explains shard-A leakage explicitly so users understand the consequence. worthless-v5sy (P3 follow-up) adds 'worthless lock --strict' which upgrades the warning to a refusal for CI/team-config use cases. The CANONICAL_KEY_VAR_RE constant defined here is reused there. Test: tests/test_cli_lock.py::TestLockFormatPreserving:: test_lock_warns_on_non_canonical_var_name. .env contains MY_OPENAI_KEY (non-canonical). Asserts: - exit_code == 0 (soft, lock proceeds) - warning output names MY_OPENAI_KEY - warning output mentions shard-A or bypass (consequence stated) - DB has the enrollment (lock proceeded despite warning) Mutation-tested: replacing the warning emit with a no-op causes the test to fail because the assertion on warning text doesn't match. Refs: worthless-8rqs (this PR), worthless-v5sy (P3 follow-up), seam 5 in worthless-8rqs design notes. * fix: address PR #127 expert review (M5) CodeRabbit's automated review on the M4 commit surfaced 10 actionable findings on PR #127. Triaged and addressed in this commit. The critical one is a security regression introduced by M1 itself. CRITICAL — anti-enumeration regression in M1 (#11) M1 hoisted the NULL base_url check above reconstruction (correct for SR-03) but returned a distinctive 503 with a relock hint. That broke the proxy's anti-enumeration contract: random alias returned uniform 401, real-but-legacy alias returned distinguishable 503. Same oracle class as worthless-bi7h's timing oracle, but content-shape rather than timing. Attacker probing aliases could distinguish "doesn't exist" from "exists but legacy." Fix: return _uniform_401() (byte-identical to unknown-alias path), log the relock hint server-side via logger.warning. Operators see the legacy-row signal in logs; the wire stays uniform. Test: tests/test_proxy_hardening.py::TestGateBeforeDecrypt:: test_null_base_url_refused_before_reconstruction strengthened to pin THREE contracts with assertions ordered call-counts-first so a regression in any one gate surfaces at the right contract: 1. SR-03: reconstruct_key + reconstruct_key_fp call_count == 0 2. Rules-engine skipped: rules_engine.evaluate call_count == 0 (CodeRabbit #13 — was previously implicit, now pinned) 3. Anti-enumeration: status + body byte-equal to unknown-alias response captured from the same proxy in the same test 4. Operator signal: caplog asserts the warning fired with the alias name (otherwise the relock hint silently regresses) Mutation-tested: replacing the M5 fix with the original M1 503 makes the test fail on the status assertion with a clean diagnostic. Quick wins folded in - providers.py: _strip_trailing_slash() + apply at parse-time and lookup-time. Trailing slash on user URLs (e.g. .../v1/) used to fail M3's registry membership gate; now resolves to the bundled entry. Honest naming — only handles trailing slash; scheme/host case-fold and default-port stripping deferred. (#10) - providers.py: user_names() helper, symmetric with bundled_names(). Pulled out so register_provider can share the abstraction. - commands/providers.py: _refuse_name_collision() helper. Bundled and user collision checks were 90% the same shape; extracted to prevent error-format drift between the two checks. (#7) - commands/providers.py: register now refuses re-registering an already-user-registered name. Without this guard, a duplicate register call appended a second [provider.<name>] table to the user TOML; tomllib raises on the next load_user() because TOML forbids duplicate tables, breaking every other providers/lock flow until the user hand-edits the file. (#7) - commands/providers.py:165: f-string typo. Error message showed literal "{name}-staging" instead of interpolating. (#6) - test_e2e.py:153: os.environ.setdefault → assignment. The locked .env MUST win over any ambient OPENAI_BASE_URL in the parent env; setdefault meant a developer with their own real BASE_URL set would silently bypass the proxy and the e2e would pass for the wrong reason. (#12) - SKILL.md: 3 sites updated. Wrap no longer owns *_BASE_URL injection post-Phase 8; lock writes per-enrollment URLs to .env and SDK reads via dotenv. (#4) Tests added - test_register_refuses_existing_user_name covers the duplicate-name guard (#7) - test_lookup_normalizes_trailing_slash covers the trailing-slash equivalence (#10) - test_null_base_url_refused_before_reconstruction grew to cover three contracts with operator-log assertion (#11 + #13) Findings filed as follow-up beads, not fixed here - #5 provider/protocol conflation (architectural, post-merge): worthless-9t74 (P1) - #8 unlock should restore original BASE_URL: already filed as worthless-6qzq (P1) before this review - #9 wrap should preserve port for child env: already filed as worthless-nvi3 (P2) before this review - #1, #2, #3 Semgrep findings: pre-existing on main, false positive (Python 3.7 importlib check on a 3.10+ codebase) or annotated with # nosec on a separate diff Full suite: 1850 passed, 10 skipped, 0 failed. Refs: PR #127, worthless-8rqs, worthless-9t74, worthless-6qzq, worthless-nvi3 * fix(cli): UTF-8 encoding on TOML reads (M6, CodeRabbit on M5) CodeRabbit's review of the M5 commit caught a latent Windows compat bug in the registry loader. Both load_bundled() and load_user() called read_text() without an explicit encoding parameter. Without it, read_text() falls back to locale.getencoding(), which on a fresh Windows install is cp1252 — not UTF-8. TOML is specified as UTF-8. Today's bundled providers.toml is pure ASCII (openai, anthropic, openrouter, groq, together, ollama) so the bug is silent on every existing CI matrix slot. The smoke-windows job has been passing by accident, not by contract. The moment a user adds a non-ASCII character to ~/.worthless/providers.toml — an i18n provider name via quoted-key syntax, an em-dash in a comment, a region parameter on a URL — the cp1252 decode either corrupts the bytes silently or raises UnicodeDecodeError. Latent now, real later. Fix: add encoding="utf-8" to both read_text() calls. Test: tests/cli/test_providers_command.py::TestProvidersRegister:: test_load_user_decodes_non_ascii_as_utf8. Test writes a TOML containing a Cyrillic provider name (via quoted-key syntax — TOML bare keys are ASCII-only) and a non-ASCII URL fragment (region=eu) to a tmp_path .worthless/providers.toml, monkey-patches HOME, calls load_user(), asserts both the Cyrillic name and the non-ASCII URL are present in the parsed result. Mutation note: a strict mutation test (revert encoding= and re-run under a non-UTF-8 locale) is inconclusive on macOS because Python 3.10+ in UTF-8 mode (PYTHONUTF8=1, default on most modern installs) shadows locale.getencoding() and read_text() decodes UTF-8 anyway. The mutation IS catchable on Windows CI where PYTHONUTF8 is not the default and locale falls back to cp1252. The CI matrix's smoke-windows job is the real validator for this contract going forward. Refs: PR #127 CodeRabbit review thread on src/worthless/cli/providers.py * fix(storage,cli): close 3 stale Semgrep alerts honestly (M7) Two real items + one stale-rule suppression. All three pre-existed on main; closing them in the same series keeps PR #127's review trail clean. After M7 the only remaining wire-in-the-grass is whichever Semgrep CI pass takes to re-scan and clear the dashboard. storage/schema.py — VACUUM INTO with operator-controlled path The migration that backs up the SQLite DB before adding shards.base_url inlines the backup path into a single-quoted SQL literal. SQLite's VACUUM INTO does not accept a parameterised path (the "VACUUM INTO ?" form is a syntax error), so the path MUST be inlined. The previous "# noqa: S608 — internal path, not user input" annotation asserted "trust us" without proving it. An operator with a single quote anywhere in their HOME (or whoever set WORTHLESS_DB_PATH) would have crashed migration mid-transaction with a SyntaxError — operator-self- pwn, not network-reachable, but a real footgun. Fix: _assert_safe_db_path() rejects ', NUL, CR, LF, TAB before the f-string interpolation. Validator runs at the top of the migration, so any unsafe path raises ValueError before any SQL fires. The nosemgrep + noqa annotations on the SQL line now point at the validator that makes them honest, instead of just suppressing. Tests: - test_assert_safe_db_path_rejects_quotes_and_control_chars pins the unit-level contract: 5 unsafe characters all raise ValueError, normal POSIX/Windows paths pass through silently. Backslash is intentionally allowed for Windows paths (legal in SQLite literals); test data carries a comment so a future contributor doesn't "fix" it. - test_migrate_base_url_column_refuses_unsafe_db_path is end-to-end: the migration fails fast on a quote-bearing path before any SQL hits the connection. Uses an in-memory SQLite (no SCHEMA setup needed). Mutation-tested: removing the validator call causes this test to fail with sqlite3.OperationalError "near 'quote': syntax error". cli/providers.py — importlib.resources Python-3.7 compat false positive Semgrep's python37-compatibility-importlib2 rule fires on any "from importlib import resources" because the module was added in Python 3.7. Project floor is requires-python = ">=3.10". The API we actually use (resources.files()) is 3.9+. Rule does not apply. Fix: nosemgrep annotation on the import line with a comment naming the project's Python floor. No code change. Refs: PR #127 — closes 3 of 3 still-open Semgrep OSS review threads (11 CodeRabbit threads were already resolved in M5/M6). * fix(cli): make load_user() resilient to malformed registry (M9) CodeRabbit's review of M7's UTF-8 fix caught two real bugs in load_user()'s "return {} on any failure mode" promise: 1. UnicodeDecodeError not in except tuple M6 added encoding="utf-8" to read_text() but missed that read_text(encoding=...) raises UnicodeDecodeError if the bytes aren't valid UTF-8 (e.g. corrupted file, Latin-1 paste from a Windows tool). The except (TOMLDecodeError, OSError) tuple didn't catch it, so a bad-bytes file crashed every providers/lock/unlock invocation instead of falling back to the empty registry. 2. provider as string crashes .items() _parse_toml_to_entries called raw.get("provider", {}).items() without checking the type. If a user typed `provider = "openrouter"` at file root (common config-file confusion vs. the [provider.name] table syntax), .items() raised AttributeError on the string. Fixes: - Added UnicodeDecodeError to load_user()'s except tuple - Added isinstance(providers, dict) guard in _parse_toml_to_entries with a logged warning naming the actual type seen Tests: - test_load_user_returns_empty_on_non_utf8_bytes — writes a TOML with a lone 0xc3 (invalid UTF-8 continuation), asserts {} not raise - test_load_user_returns_empty_on_provider_not_a_table — writes `provider = "openrouter"` at root, asserts {} not raise Both pass; both fail without their respective fixes. Refs: PR #127 CodeRabbit review thread on src/worthless/cli/providers.py post-M7 (resolved as part of this commit). * fix(tests): migrate openclaw fixtures to per-enrollment base_url (M10) P3 fixture migration deferred from the original 8rqs PR plan. CI on M9 was failing 11 of 18 OpenClaw E2E tests with "401 upstream provider error" because the proxy was hitting real api.openai.com instead of the in-stack mock — Phase 5+6 ripped the global per-provider URL overrides (WORTHLESS_UPSTREAM_*_URL env vars) but the openclaw fixture still relied on them. Migration approach: per-enrollment base_url stored at lock time. tests/openclaw/mock-upstream/app.py Adds /openai/v1/chat/completions and /anthropic/v1/messages alias routes so the registry can hold two entries on the same host:port with different protocols (registry keys are URL-based, can't have two entries on the same URL with different protocols). The /v1 routes stay for backward-compat. tests/openclaw/docker-compose.yml Removes WORTHLESS_UPSTREAM_OPENAI_URL and WORTHLESS_UPSTREAM_ANTHROPIC_URL — both no-ops post-Phase 5+6. tests/test_openclaw_e2e.py (openclaw_stack fixture) Pre-lock: register openai-mock URL in the user provider registry via `worthless providers register`. Then write a .env containing both OPENAI_API_KEY and OPENAI_BASE_URL pointing at the mock. Lock-time URL validation (M3) accepts the URL because it's now in the merged registry; lock stores that URL in the per-enrollment shards.base_url column; proxy reads encrypted.base_url at request time and forwards to the mock. tests/test_openclaw_e2e.py (openclaw_anthropic_alias fixture) Same pattern with the anthropic-mock URL and ANTHROPIC_BASE_URL. Dockerfile HOME=/data so Path.home() resolves to the writable /data volume. Without this, `worthless providers register` would fail mid-write on read_only:true containers because the user-provider registry is anchored on Path.home() / ".worthless/" — and /home/worthless is on the read-only root filesystem. The compose stack mounts /data as a writable volume, so this redirects user-config to a writable path. Affects production deploys too (any read_only:true container that uses `worthless providers register` benefits), not just tests. tests/install_fixtures/docker-compose.lock-e2e.yml Documents WORTHLESS_UPSTREAM_OPENAI_URL as no-op-on-8rqs but kept for the older PyPI worthless installed by install.sh. Once 8rqs ships to PyPI, swap to the per-enrollment pattern. install_docker bare-OS lock-e2e tests pass with this. tests/install_fixtures/lock_e2e.py Reverted my earlier register+OPENAI_BASE_URL changes. PyPI worthless doesn't have the `providers` subcommand yet, so the test fixture uses the old WORTHLESS_UPSTREAM_OPENAI_URL contract (still honored there). Comment explains the migration path post-PyPI-ship. LOCAL VALIDATION - tests/test_openclaw_e2e.py: 11 passed, 7 skipped (anthropic + opt openclaw-gateway), 0 failed in 41.7s. - tests/test_install_docker.py::test_lock_lifecycle_end_to_end[ubuntu-bare]: passed in 21.4s. - Full unit suite (tests/ minus docker): 1881 passed, 10 skipped, 10 xfailed in 114.5s. - All M-numbered fixes from the PR series remain green. Refs: PR #127, worthless-8rqs (parent feature), worthless-9t74 (provider/protocol structural separation, lands post-merge), and the M10 fixture migration that closes the P3 follow-up flagged in PR #127's "Known issues" section. * fix(tests): check .env write returncode in openclaw fixture (M11) CodeRabbit catch on M10. _write_env_to_container() returns a CompletedProcess; if the write fails (volume out-of-space, permission issue, sh: command not found inside container), the next docker_exec call attempts `worthless lock --env /tmp/.env` against a nonexistent file, fails for "missing .env" reasons, and the test asserts "Lock failed" — shadowing the actual setup failure. Two-line fix: assign the write result and assert before lock, matching the pattern already used in the openclaw_anthropic_alias fixture. Refs: PR #127 CodeRabbit thread on tests/test_openclaw_e2e.py:174, post-M10. * chore(proxy): rename "Token extraction failed" → "Usage extraction failed" (M12) Semgrep's python-logger-credential-disclosure rule pattern-matches on the word "Token" in log message strings, suspecting auth-token leakage. The flagged log call (proxy/app.py inside _do_record_spend) was using "Token" to mean LLM usage-tokens (the count-of-tokens-consumed for metering), not an auth token — neither the alias substitution (a SHA-256 fingerprint, public identifier) nor the provider substitution (a wire- protocol enum) is a credential. Pure terminology overlap, true false positive. The previous # nosemgrep annotation was correct but fragile (rule re-emits on dashboard refresh; alerts pinned to old commits don't auto-clear). Renaming the message to "Usage extraction failed" matches the function it lives in (_do_record_spend, which does USAGE extraction via extract_usage_openai/extract_usage_anthropic), removes the lexical trigger, and lets us drop the suppression annotation entirely. No behavior change. The 139 proxy + hardening + metering tests pass unchanged (no tests pin the log message literal). Refs: PR #127 — closes the last remaining Semgrep OSS alert (#21, anchored to refs/heads/main commit e9b2e8f). Annotation drop is the honest fix; renaming a log message is cheaper than maintaining a nosemgrep that gets re-flagged on every dashboard re-scan. * fix(storage): nosemgrep annotation must be IMMEDIATELY above match (M14) CI Semgrep scan on M13 surfaced 3 new alerts (#82, #85, #86) that I thought M7 had handled — but Semgrep's annotation parser requires the # nosemgrep comment to be on the SAME LINE as the matched code OR the line IMMEDIATELY ABOVE. M7's annotations were 2-3 lines above the actual match (separated by a comment block + the f-string assignment), so Semgrep's parser ignored them. Confirmed via local Semgrep run BEFORE the fix: ❯❯❱ python.lang.security.audit.formatted-sql-query.formatted-sql-query ❰❰ Blocking ❱❱ 140┆ await db.execute(vacuum_sql) ❯❯❱ python.sqlalchemy.security.sqlalchemy-execute-raw-query.sqlalchemy-execute-raw-query ❰❰ Blocking ❱❱ 140┆ await db.execute(vacuum_sql) The f-string lived on line 139 (the assignment), but Semgrep's taint analysis reported the match on line 140 (the call site where vacuum_sql is consumed). Annotations on lines 137-138 were too far. Fix: collapse the f-string back inline into the db.execute() call, then put a single combined # nosemgrep on the line directly above: # nosemgrep: formatted-sql-query, sqlalchemy-execute-raw-query await db.execute(f"VACUUM INTO '{backup_path}'") # noqa: S608 Also added a code comment explaining the immediate-line constraint so a future contributor doesn't re-split this and re-trigger the alert. Local Semgrep AFTER the fix: 0 findings, 0 blocking. The same finding class re-fired on PR #127 twice — first cleared by M7's annotation, then re-triggered after a refactor moved the f-string to a variable. The inline form is more durable: refactors that keep the call shape intact won't re-flag. Refs: PR #127 — Semgrep alerts #82, #85, #86 (post-M13 scan); same rules originally addressed in M7. * fix(cli,tests): two CodeRabbit findings on M14 (M15) 1. providers.py: register_provider used `url in bundled` (raw) but bundled keys are normalised. User passing trailing-slash URL bypassed the bundled-collision check and silently overrode the bundled entry on next load_registry. Fix: lookup_by_url + scope refusal to bundled names only. 2. test_docker_e2e.py: tightened test_wrap_injects_base_url assertion from `'127.0.0.1' in base_url` to exact-match on the alias-qualified URL so a regression dropping /{alias}/v1 surfaces here. Refs: PR #127 — CodeRabbit threads on M14. * fix(tests): bump install-docker subprocess timeouts (M16) CI runner load on GitHub Actions has grown; 240s docker-build / 360s docker-compose timeouts are now too tight for the install.sh host-matrix workflow on PR #127. Same workflow passed 5 days ago on main; nothing in 8rqs's diff slows the build (we don't touch Dockerfile.ubuntu-bare, install.sh fetches PyPI's published worthless which we haven't changed). Build does: apt-get update + ca-certs/curl/tar + install.sh (downloads uv + worthless from PyPI). Compose adds: mock-upstream build + lock_e2e.py runtime. Both are network-bound; runner load makes them variable. Bumped: 240s → 480s for build, 360s → 600s for compose. Outer job timeout is 25min so 5x headroom remains. Comments document why. This is a fixture timeout adjustment, not a 8rqs feature change. The actual lock-lifecycle test that exercises 8rqs's flow continues to use the older PyPI worthless via install.sh (same as before). * fix: 2 more CodeRabbit findings (M17) 1. providers.py: _atomic_write_text + read_text without encoding=utf-8 TOML spec mandates UTF-8 but Python defaults to platform locale. M6 fixed read_text in cli/providers.py (loader); this fixes the write side in commands/providers.py (register) and the read side for the existing-file append at line 219. Mirrors the same UTF-8 contract. 2. schema.py: _migrate_decoy_hash_column never converges on rerun The early return on column-exists also skipped the CREATE INDEX, so a DB left mid-migration (ALTER succeeded, index creation crashed) stayed broken across subsequent migrate_db() calls. Now: ALTER is column-conditional, CREATE INDEX always runs (idempotent). Both Minor severity, both real correctness gaps. No new tests — fix is small and the existing migration suite exercises both paths. Refs: PR #127 — CodeRabbit threads PRRT_kwDORnDwR85_k153 (UTF-8) and PRRT_kwDORnDwR85_k159 (index convergence).
Summary
WorthlessError(WRTLS-NNN)— users see clean error messages instead of Python tracebacksshard_amemory leak on unlock ambiguity path — bytearray is now zeroed on all exit pathsCommits (7)
7b9c4a9— lock/unlock error branch tests (compensation paths, symlink rejection, multi-enrollment)dac7d7b— wrap/up error branch tests (spawn failure, health timeout, child spawn, cleanup edge cases)545924c— Hypothesis security property tests (19 tests across 7 classes)9acfa67— quality pass addressing Brutus/Karen review gaps426a2a9— production fix: proper error handling in bootstrap, lock, unlock, up, wrapc5193f4— complete error branch coverage + bootstrap testsf2be29c— security fix: shard_a memory leak + narrowed _init_db catchTest plan
Follow-up
worthless-t7h: @error_boundary decorator + --debug flag + error message sanitization (P3 backlog)🤖 Generated with Claude Code
Summary by CodeRabbit
Bug Fixes
Tests