Skip to content

test(install): WOR-274 — chain worthless lock through post-install Docker e2e - #85

Merged
oblangatas merged 1 commit into
mainfrom
shacharm/wor-274-install-lock-e2e
Apr 21, 2026
Merged

test(install): WOR-274 — chain worthless lock through post-install Docker e2e#85
oblangatas merged 1 commit into
mainfrom
shacharm/wor-274-install-lock-e2e

Conversation

@oblangatas

@oblangatas oblangatas commented Apr 21, 2026

Copy link
Copy Markdown
Owner

Fresh-box install now proves worthless lock works end-to-end, not just --version.

TL;DR: Closes the WOR-235 AC gap by chaining install → lock → proxy → fake upstream on a bare-Ubuntu container, asserting the reconstructed real key arrives upstream.

Summary

The existing install-docker.yml job proves a non-Python Linux box can run curl|sh and end up with a worthless --version that prints. That's the floor of the AC for WOR-235, not the ceiling — users install worthless to lock keys, and nothing on CI covered that post-install. If the published worthless==0.3.0 wheel regressed the lock lifecycle, we'd ship it.

This chains the real feature on top of the existing install image:

  1. Bare-Ubuntu → curl|sh (unchanged invariant: no python/pipx/uv pre-installed).
  2. Write a fake OpenAI key to .env, run worthless lock --env .env — shard-A stays in .env, shard-B lands in the DB.
  3. Start worthless up in-container, send a request with shard-A as Bearer to the proxy's /<alias>/v1/chat/completions.
  4. Assert mock-upstream captured a request where Authorization: Bearer <real_key> — proving reconstruction from shard-A + shard-B actually happened.

What

  • tests/install_fixtures/Dockerfile.ubuntu-bare-lock-e2e — extends the bare-Ubuntu baseline, runs install.sh at build time, then layers python3 after install so the fresh-box invariant still holds during the AC check. Entrypoint is the orchestrator.
  • tests/install_fixtures/docker-compose.lock-e2e.yml — wires the installed container to mock-upstream on a private bridge; passes WORTHLESS_UPSTREAM_OPENAI_URL so the proxy talks to the fake instead of api.openai.com.
  • tests/install_fixtures/lock_e2e.py — stdlib-only orchestration. Runs inside the worthless-installed container, drives the whole chain, exits non-zero with a clear message on any step that regresses.
  • tests/test_install_docker.py — new test_bare_ubuntu_lock_lifecycle_end_to_end (marker=docker). Uses docker compose up --abort-on-container-exit --exit-code-from, captures logs for diagnosis, always tears down.
  • .dockerignore — whitelist tests/install_fixtures/ so the Dockerfile can COPY lock_e2e.py without leaking the rest of tests/ into the production image.

Why

The WOR-235 AC says "fresh machine → working install → worthless lock works". Before this PR we proved (1) and (2). Missing (3) meant a broken wheel on PyPI would ship silently — the worthless --version happy path says nothing about whether shard splitting, the proxy, or format-preserving reconstruction still work against a real-ish provider. Reuses the existing tests/openclaw/mock-upstream fake instead of building another one.

How

  • Built a new compose stack + Dockerfile pair so the baseline Dockerfile.ubuntu-bare stays minimal and the lock-e2e image can layer on orchestration tooling without weakening the fresh-box guard.
  • Orchestrator is pure stdlib — no pip install inside the image, so build time stays bounded by install.sh itself.
  • Compose tears down via docker compose down -v --remove-orphans in a finally block regardless of test outcome or timeout.
  • The install-docker workflow already triggers on tests/install_fixtures/**, so this runs automatically on every installer-touching PR.
  • Local wall time: ~2 min uncached, ~10s hot. Well under the <3 min additional runtime the ticket allowed.

Test plan

  • uv run pytest -m docker tests/test_install_docker.py (both tests) — 2 passed
  • uv run pytest tests/test_install_static.py tests/test_install_shellcheck.py tests/test_install_logic.py — 21 passed, 1 skipped
  • uv run ruff check / uv run ruff format --check clean on new files
  • Pre-commit hooks pass (ruff, codespell, bandit, gitleaks, etc.)
  • CI install-docker.yml green on this PR

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests

    • Added comprehensive end-to-end tests validating lock behavior with mock upstream infrastructure, verifying key handling and proxy functionality.
  • Chores

    • Updated Docker build and Compose configurations to support end-to-end test fixtures.

…Docker e2e

Close the WOR-235 AC gap: the existing install-Docker test only proves
`worthless --version` runs — it never exercises the product feature users
actually install worthless for. This chains the full lifecycle on a fresh
bare-Ubuntu install (curl|sh → uv → worthless), then runs
`worthless lock` → `worthless up` → a proxied request → asserts the
fake upstream received the reconstructed real key.

Reuses the existing mock-upstream (tests/openclaw/mock-upstream) instead
of building a new fake. New artifacts:
  - Dockerfile.ubuntu-bare-lock-e2e — installs worthless at build time,
    keeps the fresh-box AC guard (no python/pipx/uv before install.sh)
    and adds python3 only *after* install.sh succeeds.
  - docker-compose.lock-e2e.yml — worthless-installed + mock-upstream on
    a private bridge.
  - lock_e2e.py — stdlib-only orchestration that runs in-container.

Wall time on CI: ~2 min uncached, ~10s hot. Dockerfile.ubuntu-bare remains
the minimal AC image; this one is additive.

.dockerignore: whitelist tests/install_fixtures/ so the Dockerfile can
COPY lock_e2e.py without leaking the rest of tests/ into the image.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Apr 21, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

This PR adds end-to-end testing infrastructure for the lock lifecycle feature by introducing Docker-based fixtures and orchestration scripts that validate key splitting and reconstruction between worthless lock and worthless up with a mock upstream service.

Changes

Cohort / File(s) Summary
Docker Configuration
.dockerignore, tests/install_fixtures/Dockerfile.ubuntu-bare-lock-e2e, tests/install_fixtures/docker-compose.lock-e2e.yml
Added Docker Compose e2e test fixture with Ubuntu bare-metal Dockerfile that installs minimal dependencies, runs install.sh, and executes the lock e2e test harness. Also configured .dockerignore to include test fixtures during Docker builds.
E2E Test Orchestration
tests/install_fixtures/lock_e2e.py
New post-install orchestration script that validates lock behavior by writing a fake API key, running worthless lock --env to split the key, starting worthless up as a background proxy, sending a request with a shard, and asserting the mock upstream receives the reconstructed original key.
Test Module
tests/test_install_docker.py
Extended test suite with new end-to-end test test_bare_ubuntu_lock_lifecycle_end_to_end that launches the Docker Compose stack, validates successful exit, and ensures stack teardown.

Sequence Diagram

sequenceDiagram
    actor TestRunner as Test Runner
    participant Compose as Docker Compose
    participant WorthlessImg as worthless-installed
    participant MockUp as mock-upstream
    
    TestRunner->>Compose: docker compose up --build
    Compose->>MockUp: Start mock-upstream service
    MockUp->>MockUp: Health check polling
    Compose->>WorthlessImg: Start worthless-installed (depends_on healthy)
    WorthlessImg->>WorthlessImg: Run install.sh
    WorthlessImg->>WorthlessImg: Execute lock_e2e.py
    WorthlessImg->>WorthlessImg: Write fake OpenAI key to .env
    WorthlessImg->>WorthlessImg: Run worthless lock --env (splits key)
    WorthlessImg->>WorthlessImg: Start worthless up (background)
    WorthlessImg->>WorthlessImg: Poll proxy health until ready
    WorthlessImg->>WorthlessImg: Clear mock upstream captured headers
    WorthlessImg->>MockUp: POST /alias/v1/chat/completions with shard_a
    MockUp->>MockUp: Capture Authorization header
    WorthlessImg->>MockUp: GET /captured headers
    MockUp-->>WorthlessImg: Return captured requests with reconstructed key
    WorthlessImg->>WorthlessImg: Assert original key matches upstream Authorization
    WorthlessImg->>WorthlessImg: Terminate worthless up
    Compose-->>TestRunner: Exit code 0
    TestRunner->>Compose: docker compose down -v
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • feat: OpenClaw integration test (WOR-213) #53: Both PRs integrate Docker Compose–based end-to-end testing with a mock upstream service; this PR's fixtures set the WORTHLESS_UPSTREAM_OPENAI_URL environment variable to configure the adapter for proxying to the mock service, creating a direct workflow dependency.

Poem

🐰 A lock that splits, a proxy that mends,
Keys shattered, then whole again—
Docker builds test the dance divine,
From shard to secret, endpoint to mine! 🔑

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 37.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly and specifically describes the main change: adding a Docker e2e test that chains the lock workflow through post-install, matching the PR's core objective of verifying lock lifecycle after install.sh.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch shacharm/wor-274-install-lock-e2e

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/install_fixtures/lock_e2e.py (1)

101-108: Preserve worthless up logs for e2e failures.

Discarding stdout/stderr makes proxy startup and routing failures opaque even though the pytest wrapper dumps compose logs. Let the child inherit the container streams, or redirect to a temp file and print it on failure.

Suggested log preservation
     proxy = subprocess.Popen(  # noqa: S603, S607
         ["worthless", "up"],  # noqa: S607
-        stdout=subprocess.DEVNULL,
-        stderr=subprocess.DEVNULL,
     )
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/install_fixtures/lock_e2e.py` around lines 101 - 108, The proxy
subprocess is started with stdout/stderr discarded which hides useful startup
logs; update the subprocess.Popen call that creates proxy (the call currently
assigned to variable proxy) to either inherit the parent's stdout/stderr (so
logs appear in the test output) or redirect both streams to a temporary file,
and then when wait_until_healthy(PROXY_HEALTH_URL) fails call fail("proxy did
not become healthy within 30s") only after dumping/printing the saved logs;
ensure the cleanup still closes the temp file or terminates proxy as needed so
logs are available for the pytest failure report.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@tests/install_fixtures/lock_e2e.py`:
- Around line 101-108: The proxy subprocess is started with stdout/stderr
discarded which hides useful startup logs; update the subprocess.Popen call that
creates proxy (the call currently assigned to variable proxy) to either inherit
the parent's stdout/stderr (so logs appear in the test output) or redirect both
streams to a temporary file, and then when wait_until_healthy(PROXY_HEALTH_URL)
fails call fail("proxy did not become healthy within 30s") only after
dumping/printing the saved logs; ensure the cleanup still closes the temp file
or terminates proxy as needed so logs are available for the pytest failure
report.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 69fa4a49-1a68-4c0d-9963-7fd48ec63586

📥 Commits

Reviewing files that changed from the base of the PR and between d409a23 and 1e77a77.

📒 Files selected for processing (5)
  • .dockerignore
  • tests/install_fixtures/Dockerfile.ubuntu-bare-lock-e2e
  • tests/install_fixtures/docker-compose.lock-e2e.yml
  • tests/install_fixtures/lock_e2e.py
  • tests/test_install_docker.py

@oblangatas
oblangatas merged commit c65c2e3 into main Apr 21, 2026
35 checks passed
@oblangatas
oblangatas deleted the shacharm/wor-274-install-lock-e2e branch April 21, 2026 08:34
oblangatas added a commit that referenced this pull request Apr 21, 2026
Previously `worthless up`'s stdout/stderr were piped to DEVNULL, so when
the container-side e2e failed the outer pytest saw only "exit code 1"
with no diagnostic context. Capture to /tmp/worthless-up.log and dump
to stderr on proxy-health timeout. Addresses CodeRabbit nitpick from
PR #85.

Co-authored-by: Shachar <86647682+shachar-ug@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
oblangatas added a commit that referenced this pull request May 5, 2026
CI Semgrep scan on M13 surfaced 3 new alerts (#82, #85, #86) that I
thought M7 had handled — but Semgrep's annotation parser requires
the # nosemgrep comment to be on the SAME LINE as the matched code
OR the line IMMEDIATELY ABOVE. M7's annotations were 2-3 lines above
the actual match (separated by a comment block + the f-string
assignment), so Semgrep's parser ignored them.

Confirmed via local Semgrep run BEFORE the fix:

  ❯❯❱ python.lang.security.audit.formatted-sql-query.formatted-sql-query
       ❰❰ Blocking ❱❱
        140┆ await db.execute(vacuum_sql)
  ❯❯❱ python.sqlalchemy.security.sqlalchemy-execute-raw-query.sqlalchemy-execute-raw-query
       ❰❰ Blocking ❱❱
        140┆ await db.execute(vacuum_sql)

The f-string lived on line 139 (the assignment), but Semgrep's taint
analysis reported the match on line 140 (the call site where
vacuum_sql is consumed). Annotations on lines 137-138 were too far.

Fix: collapse the f-string back inline into the db.execute() call,
then put a single combined # nosemgrep on the line directly above:

    # nosemgrep: formatted-sql-query, sqlalchemy-execute-raw-query
    await db.execute(f"VACUUM INTO '{backup_path}'")  # noqa: S608

Also added a code comment explaining the immediate-line constraint
so a future contributor doesn't re-split this and re-trigger the
alert.

Local Semgrep AFTER the fix: 0 findings, 0 blocking.

The same finding class re-fired on PR #127 twice — first cleared by
M7's annotation, then re-triggered after a refactor moved the f-string
to a variable. The inline form is more durable: refactors that keep
the call shape intact won't re-flag.

Refs: PR #127 — Semgrep alerts #82, #85, #86 (post-M13 scan); same
rules originally addressed in M7.
oblangatas added a commit that referenced this pull request May 5, 2026
…am) (#127)

* feat(cli): bundled TOML provider registry with user override (8rqs Phase 1)

The first plumbing piece for worthless-8rqs (lock multiple LLM providers
side-by-side): a TOML registry mapping known upstream URLs to wire protocol,
so `worthless lock` can auto-detect the protocol when scanning .env files.

What ships:
- src/worthless/providers.toml — bundled with 6 entries (openai, anthropic,
  openrouter, groq, together, ollama). Practical-set decision; users can
  add more locally without touching the package.
- src/worthless/cli/providers.py — loader. Reads bundled file via
  importlib.resources, optionally merges ~/.worthless/providers.toml on
  top (user wins on URL conflict). Malformed user file logs a warning and
  falls back to bundled-only — never crashes lock.
- pyproject.toml — adds tomli to runtime deps (3.10 fallback for tomllib);
  package-data line so providers.toml ships in the wheel; deptry DEP001
  whitelist for stdlib tomllib.
- uv.lock — regenerated for the new runtime tomli dep.

TDD: 12 tests in tests/cli/test_providers_registry.py, all passing.
Mutation-tested: dropping the openai entry from providers.toml fails
test_bundled_registry_has_six_providers AND test_bundled_includes_openai
with clear diagnostics.

Phase 1 of the worthless-8rqs plan.

* feat(cli): worthless providers list/register subcommands (8rqs Phase 2)

Surface the registry to users:

- `worthless providers list` prints the merged registry as a table
  (NAME, PROTOCOL, SOURCE, URL). Source is "bundled" or "user".
- `worthless providers list --json` (via the global -j/--json flag)
  emits machine-readable JSON.
- `worthless providers register --name --url --protocol [--force]`
  appends a custom provider to ~/.worthless/providers.toml. Atomic
  write (.tmp + rename), mode 0644 (registry is public data).

Validation up front:
- name: alphanumeric + hyphen/underscore (regex)
- url: scheme in {http, https}, non-empty netloc — rejects `not-a-url`,
  `javascript:alert(1)`. Accepts `http://localhost:1234/v1` (Ollama).
- protocol: must be in {openai, anthropic}.
- name conflict with bundled → refused with hint.
- url conflict with bundled → refused unless --force.

Surface changes outside the new module:
- `cli/providers.py`: rename `_load_bundled`/`_load_user` → public
  `load_bundled`/`load_user`; add `bundled_names()` and
  `user_registry_path()` helpers used by the register command.
- `cli/errors.py`: add `ErrorCode.INVALID_INPUT = 112` for validation
  failures (no existing code matched).
- `cli/app.py`: register the new subcommand group.

TDD: 13 tests in tests/cli/test_providers_command.py, all passing.
Phase 1 tests still green (12/12). Total Phase 1+2 coverage: 25 tests.

Phase 2 of the worthless-8rqs plan.

* feat(storage): add shards.base_url with VACUUM INTO backup, no backfill (8rqs Phase 3)

The DB-side groundwork for per-enrollment routing. After this commit the
schema has the column; readers will start using it in Phase 4.

What ships:
- src/worthless/storage/schema.py:
  * SCHEMA gains `base_url TEXT` on the `shards` table (nullable).
  * migrate_db gains a guarded ALTER for the column.
  * BEFORE the ALTER fires (and only the first time), the DB is snapshotted
    via SQLite's VACUUM INTO 'db.sqlite.bak.<timestamp>' — a clean WAL-aware
    copy that gives the user a rollback option if anything goes sideways
    on a 0-user PR landing.
  * NO backfill. Brutus's catch from plan time: backfilling NULL → per-
    provider default would silently mis-route an OpenAI-protocol enrollment
    that pointed at OpenRouter via the old WORTHLESS_UPSTREAM_OPENAI_URL
    workaround. Phase 4's reader will refuse to read a NULL row with a
    clear "predates upstream URL storage; run worthless relock <name>"
    error, forcing explicit re-lock.

Tests (4 new in tests/test_storage.py):
- test_fresh_db_has_base_url_column — init_db creates the column from CREATE
- test_migrate_adds_base_url_column — pre-8rqs DB gets the column
- test_migrate_base_url_idempotent — running migrate twice is safe
- test_migrate_creates_backup_before_altering — backup file appears
- test_migrate_no_backup_when_already_migrated — no spam on idle migrate

All 28 storage tests still green.

Mutation-tested: removing the VACUUM INTO line makes
test_migrate_creates_backup_before_altering fail with "no pre-migration
backup created in <tmp>; existing files: [...]".

Phase 3 of the worthless-8rqs plan.

* refactor(storage): per-enrollment routing in repo API (8rqs Phase 4)

The data layer now carries base_url end-to-end. Phases 5-6 will start
USING this; Phase 4 just plumbs it through.

Surface changes:
- EncryptedShard NamedTuple gains `base_url: str | None = None` (additive,
  default keeps every existing constructor working — verified across
  10+ call sites in tests + cli/commands/unlock.py).
- __repr__ updated to include base_url (alongside the existing length-
  redacted byte fields).
- ShardRepository.store(..., base_url=None) — new kwarg, INSERT plumbed.
- ShardRepository.store_enrolled(..., base_url=None) — new kwarg, INSERT
  plumbed (the path Phase 7's lock command will use).
- ShardRepository.fetch_encrypted SELECT now includes base_url; the
  returned EncryptedShard.base_url is None for legacy rows.

Renamed (no shim): list_aliases_with_provider → list_aliases_with_routing.
- Returns 4-tuples (alias, var_name, base_url, protocol).
- Now JOINs shards × enrollments so callers get var_name in the same
  query (used by Phase 8's wrap rewrite).
- Single existing caller (cli/commands/wrap.py:_list_enrolled_aliases)
  updated in this commit; for now slices the 4-tuple back to (alias,
  protocol) so existing _build_child_env / _PROVIDER_ENV_MAP logic keeps
  working until Phase 8 deletes that path.

Tests:
- 3 existing list_aliases_with_provider tests renamed + adapted to the
  new 4-tuple shape.
- 2 new tests pin the base_url roundtrip:
  test_store_enrolled_with_base_url_roundtrips,
  test_store_enrolled_without_base_url_returns_none.
- 1 new test pins legacy null behaviour:
  test_list_aliases_with_routing_legacy_row_has_null_base_url.

31 storage tests pass (was 28, +3 net).
92 storage+cli tests pass with no regressions.
169 storage+cli+security+proxy_hardening tests pass — confirms the
EncryptedShard field-add is non-breaking for all existing consumers.

Phase 4 of the worthless-8rqs plan.

* feat(routing): per-enrollment base_url end-to-end (8rqs Phase 5+6)

Adapters and proxy now thread the upstream URL per request. The legacy
WORTHLESS_UPSTREAM_OPENAI_URL / _ANTHROPIC_URL env vars are GONE — no
back-compat (no users).

Adapter signature change (Phase 5):
- adapters/types.py: ProviderAdapter Protocol gains required base_url kwarg.
- adapters/openai.py: drop module-level UPSTREAM_URL constant + os.environ
  read; prepare_request(*, ..., base_url) computes
  f"{base_url.rstrip('/')}/chat/completions".
- adapters/anthropic.py: same shape; appends "/messages".

Proxy plumbing (Phase 6):
- proxy/app.py: after fetch_encrypted, if encrypted.base_url is None
  (legacy row predating 8rqs), return 503 with hint
  "alias <name> predates upstream URL storage; run 'worthless relock'
  to re-enroll with --base-url" rather than fall back to a possibly-wrong
  default. Otherwise pass base_url=encrypted.base_url to prepare_request.

Test surface updates (mechanical, mostly bulk-applied):
- tests/test_adapters.py: 13 prepare_request call sites updated +
  TestPerEnrollmentBaseURL class added (6 new tests pinning the contract).
- tests/test_adapter_bytearray.py: 5 sites updated.
- tests/test_properties.py: 4 sites updated.
- tests/test_contract.py: replace UPSTREAM_URL monkey-patches (test
  pollution risk under xdist) with proper enrollment-time base_url
  pointing at the mock upstream. Removed `import _oai_mod, _anth_mod`.
- tests/test_proxy*.py: 8 store() fixture sites updated with
  base_url=https://api.openai.com/v1 so per-enrollment routing kicks in.
- tests/test_error_metering_and_hardening.py: 1 site.
- SKILL.md: documents the new `worthless providers list/register`
  subcommand group (was an undocumented-commands-drift test failure).

Mutation-tested for the env-rip:
- Re-add `os.environ.get("WORTHLESS_UPSTREAM_OPENAI_URL", ...)` to
  adapters/openai.py and run TestPerEnrollmentBaseURL::test_env_var_has_no_effect_on_openai
  with the env var set to https://attacker.example/v1.
- Test FAILS with "'attacker' is contained here:" — exactly the
  regression we want to catch.

Mutation-tested for the proxy refusal of NULL base_url: covered by the
test_contract.py update — when an enrollment has base_url=None the proxy
would return 503 to the contract test, breaking it; the test passes only
because we now enroll WITH base_url.

Full suite: 1846 passed, 10 skipped, 0 failures.

Phases 5+6 of the worthless-8rqs plan, combined into one commit because
the adapter signature change forces the proxy call site update in the
same atomic change.

* feat(cli): lock reads per-enrollment base_url from .env (8rqs Phase 7)

The user-facing piece. Multi-provider .env now flows end-to-end:

1. User has OPENROUTER_API_KEY + OPENROUTER_BASE_URL in .env
2. worthless lock reads BOTH, stores the URL in DB, rewrites both VALUES
   in place (var names sacred — OPENROUTER_BASE_URL stays
   OPENROUTER_BASE_URL, never replaced with OPENAI_BASE_URL)
3. SDK auto-config reads the rewritten .env, sends to local proxy at the
   per-alias URL, proxy reconstructs and forwards upstream

Surface changes (cli/commands/lock.py):
- _derive_base_url_var(var_name, provider): preserves user naming
  (OPENROUTER_API_KEY → OPENROUTER_BASE_URL); falls back to
  _PROVIDER_ENV_MAP when var_name doesn't match the conventional shape.
- _resolve_upstream_base_url(base_url_var, env_values, provider):
  prefers user's explicit *_BASE_URL value; falls back to bundled
  registry default for the provider name.
- _PlannedUpdate gains base_url_var: str = "" (carries the derived var
  name through pass-1 → batch_rewrite for atomic single-call rewrite).
- _pass1_db_writes accepts env_values and threads base_url= into both
  store_enrolled call sites (re-enroll branch + fresh-enroll branch).
- _batch_rewrite uses p.base_url_var per-key. When the var already
  exists in .env, OVERWRITE the value (upstream is captured in DB);
  when missing, ADD with stderr notice
  ("worthless: added OPENAI_BASE_URL=... to .env (was missing)").
- _enroll_single (no-.env path) falls back to registry default.

Surface change (cli/providers.py):
- lookup_by_name(name) walks the merged registry by name. Used by
  lock.py to resolve provider → URL when no user var is set.

Tests:
- tests/test_cli_lock.py adds test_lock_reads_existing_base_url_from_env:
  asserts (a) DB row stores https://openrouter.ai/api/v1, (b) .env
  retains OPENROUTER_BASE_URL (NOT OPENAI_BASE_URL), (c) value points
  at local proxy URL containing the alias.
- 7 existing TestLockFormatPreserving tests still pass — the
  auto-create-when-missing path for OpenAI keys without BASE_URL is
  unchanged.

Note: re-applied in dedicated worktree at worthless-8rqs/ after a
concurrent session repeatedly switched the main checkout's branch
mid-edit and discarded uncommitted Phase 7 work.

Phase 7 of the worthless-8rqs plan.

* refactor(cli): wrap delegates BASE_URL ownership to lock (8rqs Phase 8)

Pre-8rqs, ``worthless wrap`` synthesised ``OPENAI_BASE_URL=
http://127.0.0.1:PORT/<alias>/v1`` into the child process env, hardcoded to
``OPENAI_*`` / ``ANTHROPIC_*`` regardless of what env-var name the user
actually used in their .env. After 8rqs Phase 7, ``worthless lock`` writes
the right local-proxy URL into the user's own variable name (e.g.
``OPENROUTER_BASE_URL``, ``GROQ_BASE_URL``) at lock time. Wrap injecting on
top of that is double-work that also breaks for non-canonical providers.

Surface changes:
- ``cli/commands/wrap.py``: deleted module-level ``_PROVIDER_ENV_MAP``;
  ``_build_child_env`` is now a passthrough (returns ``dict(os.environ)``);
  signature kept for test compatibility but ``port`` and ``aliases`` are
  ignored. Module docstring updated to reflect the new contract.
- ``cli/commands/lock.py``: pulled the small ``_PROVIDER_ENV_MAP``
  constant in-house (still needed for the fallback in
  ``_derive_base_url_var`` when var_name doesn't match ``*_API_KEY``).
- ``cli/commands/unlock.py``: changed import from ``wrap`` to ``lock``
  (the dict moved). No behaviour change in unlock.

Tests:
- ``tests/test_cli_wrap.py``:
  - Deleted obsolete tests asserting injection
    (``test_child_env_anthropic``, ``test_child_env_multiple_providers``,
    ``test_unknown_provider_skipped``, ``test_multi_alias_same_provider_*``).
  - Added ``test_child_env_no_injection``: with parent env clean, child
    env has NONE of OPENAI/ANTHROPIC/OPENROUTER_BASE_URL.
  - Added ``test_parent_baseurl_passes_through``: if parent has
    OPENROUTER_BASE_URL set (typical post-lock state), wrap forwards it
    unchanged.
- ``tests/test_e2e.py::test_wrap_real_proxy_transit``: child script
  updated to read .env via WORTHLESS_E2E_ENV_PATH (mimics how SDKs use
  ``python-dotenv``). The post-8rqs flow IS lock writes .env → SDK reads
  .env, so the test now exercises the real production path.

Result: 1844 passing, 10 skipped, 0 failures (full suite excl docker-e2e).

Note: re-applied in dedicated worktree at worthless-8rqs/ after concurrent
sessions repeatedly switched the main checkout's branch mid-edit.

Phase 8 of the worthless-8rqs plan.

* fix(scanner): lower entropy threshold 4.5 → 3.9 to admit OpenRouter keys (8rqs prereq)

Real OpenRouter keys have entropy ~4.118 — under the historical 4.5
threshold, scan + lock filter them as 'placeholders' and refuse to
enroll, breaking the entire 8rqs multi-provider story.

Mirrors PR #116 commit f087180 which fixed this on the WOR-306 sidecar
epic. That epic hasn't merged to main yet, so the branch this PR is
built on (off main 582a64f) still has the buggy 4.5 threshold. Bumping
inline so 8rqs can ship; once the epic merges to main, this line will
be identical and the merge a no-op.

The same placeholder values that were verified to fall below 3.9 in
PR #116 stay rejected (sk-your-key-here: 3.03, sk-aaaa: 0.88,
WRTLS-decoy: 3.63, sk-PLACEHOLDER: 3.74).

* fix(security): hoist NULL base_url check above reconstruction (SR-03)

Blocker M1 from PR #127 expert review. The 8rqs Phase 6 commit
(eba949e) placed the 'if encrypted.base_url is None: refuse' guard at
proxy/app.py line 382 — AFTER both rules_engine.evaluate (line 329)
AND key reconstruction (lines 363/372). A legacy row missing base_url
triggered full key materialisation in memory before being refused with
503. That violates SR-03 (gate before reconstruct).

This commit moves the guard to right after fetch_encrypted at line
~313, BEFORE shard_a extraction, body read, rules engine, and
reconstruction. The denial path no longer touches key material.

The minimum 6-LOC inline move is intentional. worthless-2pdi (P1
follow-up bead, blocked-by:worthless-8rqs) will promote this guard
into a structural validate_encrypted_row() helper covering ALL
row-shape denials (NULL base_url, missing prefix/charset, unknown
adapter, malformed protocol) — see seam 1 in worthless-8rqs design
notes. The hoist here is forward-compatible with that helper.

Test: tests/test_proxy_hardening.py::TestGateBeforeDecrypt::
test_null_base_url_refused_before_reconstruction. Mocks
worthless.proxy.app.reconstruct_key + reconstruct_key_fp; injects a
row with base_url=None via fetch_encrypted monkey-patch; sends a real
request through the proxy; asserts response 503 AND
mock_reconstruct.call_count == 0 AND mock_reconstruct_fp.call_count
== 0.

Mutation-tested: replacing the hoisted condition with `if False and
encrypted.base_url is None` causes the test to fail (reconstruct_key_fp
gets called downstream), proving the test pins the new ordering.

122 proxy tests pass.

Refs: worthless-8rqs (this PR), worthless-2pdi (root closure follow-up)

* fix(proxy): strip Content-Encoding/Content-Length on decompressed responses

Blocker M2 from PR #127 expert review. The proxy reads upstream response
bodies via httpx aread()/aiter_bytes(), which auto-decompresses gzip per
the Content-Encoding header. The decompressed bytes were then forwarded
to the SDK with the ORIGINAL Content-Encoding: gzip header still attached
— so SDKs (which trust the header) tried to gunzip plain JSON and got
'Error -3 while decompressing data: incorrect header check'. The PR #127
live-smoke required setting Accept-Encoding: identity on every request
to bypass — meaning default SDK calls were broken.

This commit drops Content-Encoding and Content-Length from forwarded
response headers in adapters/types.py::relay_response. The body is
already decompressed by httpx; without those headers, SDKs treat the
response as plain JSON and parse correctly.

Test: tests/test_proxy_hardening.py::TestGateBeforeDecrypt::
test_proxy_response_pipe_is_consistent_with_gzip_upstream. respx mocks
upstream returning gzipped JSON with Content-Encoding: gzip; client
reads the proxy's response and asserts internal consistency (either
header gone + decompressed body, or header present + still-gzipped body).
Without the fix, httpx's auto-decoder raises DecodingError before the
test can even read the body — same error class users hit.

Mutation-tested: reverting the _filter_response_headers call to
dict(response.headers) reproduces the exact DecodingError, proving
the test pins the user-visible bug.

This is the M2 minimum (~5 LOC). The P2 follow-up bead worthless-yo9o
deepens to true byte-transparent pipe (aiter_raw + async metering
reconciliation off the response path) — see seam 3 in worthless-8rqs
design notes.

169 proxy + adapter + contract tests pass.

Refs: worthless-8rqs (this PR), worthless-yo9o (deeper pipe follow-up),
      worthless-wl4p (SR-11 byte-transparency rule follow-up)

* fix(cli): lock refuses unregistered upstream URLs (Blocker #1)

Blocker M3 from PR #127 expert review. Pre-fix, lock pulled the user's
*_BASE_URL value from .env into the DB unchanged. An attacker who can
write to .env (in-scope per project threat model) could set
OPENROUTER_BASE_URL=https://attacker.example/v1 and the proxy would
forward reconstructed shard-A as Bearer to attacker.example —
credential exfiltration channel directly contradicting the per-enrollment
routing claim of PR #127.

This commit adds a registry-membership check on the user-supplied URL in
_resolve_upstream_base_url. If the URL is not in the merged registry
(bundled providers.toml + ~/.worthless/providers.toml user override),
lock refuses with WorthlessError(INVALID_INPUT) and a hint to register:
  'worthless providers register --name <n> --url <url> --protocol ...'

The DB transaction never starts on the refused path — no enrollment
created, no shard split.

Test: tests/test_cli_lock.py::TestLockFormatPreserving::
test_lock_refuses_unregistered_attacker_base_url. Sets OPENROUTER_BASE_URL=
https://attacker.example/v1 in .env, asserts: (a) exit_code != 0, (b)
error message names the rejected URL, (c) error message hints at
'worthless providers register', (d) DB has zero enrollments after refusal.

Mutation-tested: replacing the registry-check condition with `if False`
allows the attacker URL through and fails the test.

This is M3 minimum (15 LOC + 1 test). The follow-up beads deepen the
defense: worthless-rzi1 (P1) adds per-request DB-tamper re-validation
(closes the post-lock variant); worthless-8fbg (P1) hardens _validate_url
itself (rejects RFC1918/loopback unless registry-marked internal=true);
worthless-99ox (P1, folded from P2) adds adapter allowed_hosts for
defense-in-depth. See seam 2 in worthless-8rqs design notes.

72 cli/lock + providers tests pass.

Refs: worthless-8rqs (this PR), worthless-rzi1 + worthless-8fbg + worthless-99ox
(P1 follow-ups), worthless-k5tk (SR-10 semgrep enforcement)

* fix(cli): warn on non-canonical API key var name (Blocker #2)

Blocker M4 from PR #127 expert review (product-manager). User's .env
may contain non-canonical API-key vars like MY_OPENAI_KEY that don't
match the <PROVIDER>_API_KEY convention. Brutus surfaced a narrow
proxy-bypass scenario: the app reads MY_OPENAI_KEY explicitly and
constructs an SDK client without base_url= override; the SDK falls
through to api.openai.com and shard-A leaks on the wire instead of
hitting the proxy.

Per product-manager review the right behavior is a SOFT warning (not
a refusal). The warning names the var, points at the BASE_URL that
lock will set, and explains shard-A leakage explicitly so users
understand the consequence. worthless-v5sy (P3 follow-up) adds
'worthless lock --strict' which upgrades the warning to a refusal
for CI/team-config use cases. The CANONICAL_KEY_VAR_RE constant
defined here is reused there.

Test: tests/test_cli_lock.py::TestLockFormatPreserving::
test_lock_warns_on_non_canonical_var_name.

.env contains MY_OPENAI_KEY (non-canonical). Asserts:
- exit_code == 0 (soft, lock proceeds)
- warning output names MY_OPENAI_KEY
- warning output mentions shard-A or bypass (consequence stated)
- DB has the enrollment (lock proceeded despite warning)

Mutation-tested: replacing the warning emit with a no-op causes the
test to fail because the assertion on warning text doesn't match.

Refs: worthless-8rqs (this PR), worthless-v5sy (P3 follow-up),
seam 5 in worthless-8rqs design notes.

* fix: address PR #127 expert review (M5)

CodeRabbit's automated review on the M4 commit surfaced 10 actionable
findings on PR #127. Triaged and addressed in this commit. The
critical one is a security regression introduced by M1 itself.

CRITICAL — anti-enumeration regression in M1 (#11)

M1 hoisted the NULL base_url check above reconstruction (correct for
SR-03) but returned a distinctive 503 with a relock hint. That broke
the proxy's anti-enumeration contract: random alias returned uniform
401, real-but-legacy alias returned distinguishable 503. Same oracle
class as worthless-bi7h's timing oracle, but content-shape rather
than timing. Attacker probing aliases could distinguish "doesn't
exist" from "exists but legacy."

Fix: return _uniform_401() (byte-identical to unknown-alias path),
log the relock hint server-side via logger.warning. Operators see
the legacy-row signal in logs; the wire stays uniform.

Test: tests/test_proxy_hardening.py::TestGateBeforeDecrypt::
test_null_base_url_refused_before_reconstruction strengthened to
pin THREE contracts with assertions ordered call-counts-first so a
regression in any one gate surfaces at the right contract:

  1. SR-03: reconstruct_key + reconstruct_key_fp call_count == 0
  2. Rules-engine skipped: rules_engine.evaluate call_count == 0
     (CodeRabbit #13 — was previously implicit, now pinned)
  3. Anti-enumeration: status + body byte-equal to unknown-alias
     response captured from the same proxy in the same test
  4. Operator signal: caplog asserts the warning fired with the
     alias name (otherwise the relock hint silently regresses)

Mutation-tested: replacing the M5 fix with the original M1 503 makes
the test fail on the status assertion with a clean diagnostic.

Quick wins folded in

- providers.py: _strip_trailing_slash() + apply at parse-time and
  lookup-time. Trailing slash on user URLs (e.g. .../v1/) used to
  fail M3's registry membership gate; now resolves to the bundled
  entry. Honest naming — only handles trailing slash; scheme/host
  case-fold and default-port stripping deferred. (#10)

- providers.py: user_names() helper, symmetric with bundled_names().
  Pulled out so register_provider can share the abstraction.

- commands/providers.py: _refuse_name_collision() helper. Bundled
  and user collision checks were 90% the same shape; extracted to
  prevent error-format drift between the two checks. (#7)

- commands/providers.py: register now refuses re-registering an
  already-user-registered name. Without this guard, a duplicate
  register call appended a second [provider.<name>] table to the
  user TOML; tomllib raises on the next load_user() because TOML
  forbids duplicate tables, breaking every other providers/lock
  flow until the user hand-edits the file. (#7)

- commands/providers.py:165: f-string typo. Error message showed
  literal "{name}-staging" instead of interpolating. (#6)

- test_e2e.py:153: os.environ.setdefault → assignment. The locked
  .env MUST win over any ambient OPENAI_BASE_URL in the parent env;
  setdefault meant a developer with their own real BASE_URL set
  would silently bypass the proxy and the e2e would pass for the
  wrong reason. (#12)

- SKILL.md: 3 sites updated. Wrap no longer owns *_BASE_URL
  injection post-Phase 8; lock writes per-enrollment URLs to .env
  and SDK reads via dotenv. (#4)

Tests added

- test_register_refuses_existing_user_name covers the duplicate-name
  guard (#7)
- test_lookup_normalizes_trailing_slash covers the trailing-slash
  equivalence (#10)
- test_null_base_url_refused_before_reconstruction grew to cover
  three contracts with operator-log assertion (#11 + #13)

Findings filed as follow-up beads, not fixed here

- #5 provider/protocol conflation (architectural, post-merge):
  worthless-9t74 (P1)
- #8 unlock should restore original BASE_URL: already filed as
  worthless-6qzq (P1) before this review
- #9 wrap should preserve port for child env: already filed as
  worthless-nvi3 (P2) before this review
- #1, #2, #3 Semgrep findings: pre-existing on main, false positive
  (Python 3.7 importlib check on a 3.10+ codebase) or annotated
  with # nosec on a separate diff

Full suite: 1850 passed, 10 skipped, 0 failed.
Refs: PR #127, worthless-8rqs, worthless-9t74, worthless-6qzq,
worthless-nvi3

* fix(cli): UTF-8 encoding on TOML reads (M6, CodeRabbit on M5)

CodeRabbit's review of the M5 commit caught a latent Windows compat
bug in the registry loader. Both load_bundled() and load_user() called
read_text() without an explicit encoding parameter. Without it,
read_text() falls back to locale.getencoding(), which on a fresh
Windows install is cp1252 — not UTF-8. TOML is specified as UTF-8.

Today's bundled providers.toml is pure ASCII (openai, anthropic,
openrouter, groq, together, ollama) so the bug is silent on every
existing CI matrix slot. The smoke-windows job has been passing by
accident, not by contract. The moment a user adds a non-ASCII
character to ~/.worthless/providers.toml — an i18n provider name via
quoted-key syntax, an em-dash in a comment, a region parameter on a
URL — the cp1252 decode either corrupts the bytes silently or raises
UnicodeDecodeError. Latent now, real later.

Fix: add encoding="utf-8" to both read_text() calls.

Test: tests/cli/test_providers_command.py::TestProvidersRegister::
test_load_user_decodes_non_ascii_as_utf8.

Test writes a TOML containing a Cyrillic provider name (via quoted-key
syntax — TOML bare keys are ASCII-only) and a non-ASCII URL fragment
(region=eu) to a tmp_path .worthless/providers.toml, monkey-patches
HOME, calls load_user(), asserts both the Cyrillic name and the
non-ASCII URL are present in the parsed result.

Mutation note: a strict mutation test (revert encoding= and re-run
under a non-UTF-8 locale) is inconclusive on macOS because Python
3.10+ in UTF-8 mode (PYTHONUTF8=1, default on most modern installs)
shadows locale.getencoding() and read_text() decodes UTF-8 anyway.
The mutation IS catchable on Windows CI where PYTHONUTF8 is not the
default and locale falls back to cp1252. The CI matrix's
smoke-windows job is the real validator for this contract going
forward.

Refs: PR #127 CodeRabbit review thread on src/worthless/cli/providers.py

* fix(storage,cli): close 3 stale Semgrep alerts honestly (M7)

Two real items + one stale-rule suppression. All three pre-existed on
main; closing them in the same series keeps PR #127's review trail
clean. After M7 the only remaining wire-in-the-grass is whichever
Semgrep CI pass takes to re-scan and clear the dashboard.

storage/schema.py — VACUUM INTO with operator-controlled path

The migration that backs up the SQLite DB before adding shards.base_url
inlines the backup path into a single-quoted SQL literal. SQLite's
VACUUM INTO does not accept a parameterised path (the "VACUUM INTO ?"
form is a syntax error), so the path MUST be inlined. The previous
"# noqa: S608 — internal path, not user input" annotation asserted
"trust us" without proving it. An operator with a single quote
anywhere in their HOME (or whoever set WORTHLESS_DB_PATH) would have
crashed migration mid-transaction with a SyntaxError — operator-self-
pwn, not network-reachable, but a real footgun.

Fix: _assert_safe_db_path() rejects ', NUL, CR, LF, TAB before the
f-string interpolation. Validator runs at the top of the migration,
so any unsafe path raises ValueError before any SQL fires. The
nosemgrep + noqa annotations on the SQL line now point at the
validator that makes them honest, instead of just suppressing.

Tests:
- test_assert_safe_db_path_rejects_quotes_and_control_chars pins the
  unit-level contract: 5 unsafe characters all raise ValueError, normal
  POSIX/Windows paths pass through silently. Backslash is intentionally
  allowed for Windows paths (legal in SQLite literals); test data
  carries a comment so a future contributor doesn't "fix" it.
- test_migrate_base_url_column_refuses_unsafe_db_path is end-to-end:
  the migration fails fast on a quote-bearing path before any SQL hits
  the connection. Uses an in-memory SQLite (no SCHEMA setup needed).
  Mutation-tested: removing the validator call causes this test to
  fail with sqlite3.OperationalError "near 'quote': syntax error".

cli/providers.py — importlib.resources Python-3.7 compat false positive

Semgrep's python37-compatibility-importlib2 rule fires on any
"from importlib import resources" because the module was added in
Python 3.7. Project floor is requires-python = ">=3.10". The API we
actually use (resources.files()) is 3.9+. Rule does not apply.

Fix: nosemgrep annotation on the import line with a comment naming
the project's Python floor. No code change.

Refs: PR #127 — closes 3 of 3 still-open Semgrep OSS review threads
(11 CodeRabbit threads were already resolved in M5/M6).

* fix(cli): make load_user() resilient to malformed registry (M9)

CodeRabbit's review of M7's UTF-8 fix caught two real bugs in
load_user()'s "return {} on any failure mode" promise:

1. UnicodeDecodeError not in except tuple
   M6 added encoding="utf-8" to read_text() but missed that
   read_text(encoding=...) raises UnicodeDecodeError if the bytes
   aren't valid UTF-8 (e.g. corrupted file, Latin-1 paste from a
   Windows tool). The except (TOMLDecodeError, OSError) tuple didn't
   catch it, so a bad-bytes file crashed every providers/lock/unlock
   invocation instead of falling back to the empty registry.

2. provider as string crashes .items()
   _parse_toml_to_entries called raw.get("provider", {}).items()
   without checking the type. If a user typed `provider = "openrouter"`
   at file root (common config-file confusion vs. the [provider.name]
   table syntax), .items() raised AttributeError on the string.

Fixes:
- Added UnicodeDecodeError to load_user()'s except tuple
- Added isinstance(providers, dict) guard in _parse_toml_to_entries
  with a logged warning naming the actual type seen

Tests:
- test_load_user_returns_empty_on_non_utf8_bytes — writes a TOML
  with a lone 0xc3 (invalid UTF-8 continuation), asserts {} not raise
- test_load_user_returns_empty_on_provider_not_a_table — writes
  `provider = "openrouter"` at root, asserts {} not raise

Both pass; both fail without their respective fixes.

Refs: PR #127 CodeRabbit review thread on src/worthless/cli/providers.py
post-M7 (resolved as part of this commit).

* fix(tests): migrate openclaw fixtures to per-enrollment base_url (M10)

P3 fixture migration deferred from the original 8rqs PR plan. CI on
M9 was failing 11 of 18 OpenClaw E2E tests with "401 upstream provider
error" because the proxy was hitting real api.openai.com instead of
the in-stack mock — Phase 5+6 ripped the global per-provider URL
overrides (WORTHLESS_UPSTREAM_*_URL env vars) but the openclaw fixture
still relied on them.

Migration approach: per-enrollment base_url stored at lock time.

tests/openclaw/mock-upstream/app.py
  Adds /openai/v1/chat/completions and /anthropic/v1/messages alias
  routes so the registry can hold two entries on the same host:port
  with different protocols (registry keys are URL-based, can't have
  two entries on the same URL with different protocols). The /v1
  routes stay for backward-compat.

tests/openclaw/docker-compose.yml
  Removes WORTHLESS_UPSTREAM_OPENAI_URL and
  WORTHLESS_UPSTREAM_ANTHROPIC_URL — both no-ops post-Phase 5+6.

tests/test_openclaw_e2e.py (openclaw_stack fixture)
  Pre-lock: register openai-mock URL in the user provider registry
  via `worthless providers register`. Then write a .env containing
  both OPENAI_API_KEY and OPENAI_BASE_URL pointing at the mock.
  Lock-time URL validation (M3) accepts the URL because it's now in
  the merged registry; lock stores that URL in the per-enrollment
  shards.base_url column; proxy reads encrypted.base_url at request
  time and forwards to the mock.

tests/test_openclaw_e2e.py (openclaw_anthropic_alias fixture)
  Same pattern with the anthropic-mock URL and ANTHROPIC_BASE_URL.

Dockerfile
  HOME=/data so Path.home() resolves to the writable /data volume.
  Without this, `worthless providers register` would fail mid-write
  on read_only:true containers because the user-provider registry is
  anchored on Path.home() / ".worthless/" — and /home/worthless is on
  the read-only root filesystem. The compose stack mounts /data as a
  writable volume, so this redirects user-config to a writable path.
  Affects production deploys too (any read_only:true container that
  uses `worthless providers register` benefits), not just tests.

tests/install_fixtures/docker-compose.lock-e2e.yml
  Documents WORTHLESS_UPSTREAM_OPENAI_URL as no-op-on-8rqs but kept
  for the older PyPI worthless installed by install.sh. Once 8rqs
  ships to PyPI, swap to the per-enrollment pattern. install_docker
  bare-OS lock-e2e tests pass with this.

tests/install_fixtures/lock_e2e.py
  Reverted my earlier register+OPENAI_BASE_URL changes. PyPI worthless
  doesn't have the `providers` subcommand yet, so the test fixture
  uses the old WORTHLESS_UPSTREAM_OPENAI_URL contract (still honored
  there). Comment explains the migration path post-PyPI-ship.

LOCAL VALIDATION

- tests/test_openclaw_e2e.py: 11 passed, 7 skipped (anthropic + opt
  openclaw-gateway), 0 failed in 41.7s.
- tests/test_install_docker.py::test_lock_lifecycle_end_to_end[ubuntu-bare]:
  passed in 21.4s.
- Full unit suite (tests/ minus docker): 1881 passed, 10 skipped,
  10 xfailed in 114.5s.
- All M-numbered fixes from the PR series remain green.

Refs: PR #127, worthless-8rqs (parent feature), worthless-9t74
(provider/protocol structural separation, lands post-merge), and the
M10 fixture migration that closes the P3 follow-up flagged in PR
#127's "Known issues" section.

* fix(tests): check .env write returncode in openclaw fixture (M11)

CodeRabbit catch on M10. _write_env_to_container() returns a
CompletedProcess; if the write fails (volume out-of-space, permission
issue, sh: command not found inside container), the next docker_exec
call attempts `worthless lock --env /tmp/.env` against a nonexistent
file, fails for "missing .env" reasons, and the test asserts "Lock
failed" — shadowing the actual setup failure.

Two-line fix: assign the write result and assert before lock, matching
the pattern already used in the openclaw_anthropic_alias fixture.

Refs: PR #127 CodeRabbit thread on tests/test_openclaw_e2e.py:174,
post-M10.

* chore(proxy): rename "Token extraction failed" → "Usage extraction failed" (M12)

Semgrep's python-logger-credential-disclosure rule pattern-matches on
the word "Token" in log message strings, suspecting auth-token leakage.
The flagged log call (proxy/app.py inside _do_record_spend) was using
"Token" to mean LLM usage-tokens (the count-of-tokens-consumed for
metering), not an auth token — neither the alias substitution (a SHA-256
fingerprint, public identifier) nor the provider substitution (a wire-
protocol enum) is a credential. Pure terminology overlap, true false
positive.

The previous # nosemgrep annotation was correct but fragile (rule
re-emits on dashboard refresh; alerts pinned to old commits don't
auto-clear). Renaming the message to "Usage extraction failed" matches
the function it lives in (_do_record_spend, which does USAGE extraction
via extract_usage_openai/extract_usage_anthropic), removes the lexical
trigger, and lets us drop the suppression annotation entirely.

No behavior change. The 139 proxy + hardening + metering tests pass
unchanged (no tests pin the log message literal).

Refs: PR #127 — closes the last remaining Semgrep OSS alert (#21,
anchored to refs/heads/main commit e9b2e8f). Annotation drop is the
honest fix; renaming a log message is cheaper than maintaining a
nosemgrep that gets re-flagged on every dashboard re-scan.

* fix(storage): nosemgrep annotation must be IMMEDIATELY above match (M14)

CI Semgrep scan on M13 surfaced 3 new alerts (#82, #85, #86) that I
thought M7 had handled — but Semgrep's annotation parser requires
the # nosemgrep comment to be on the SAME LINE as the matched code
OR the line IMMEDIATELY ABOVE. M7's annotations were 2-3 lines above
the actual match (separated by a comment block + the f-string
assignment), so Semgrep's parser ignored them.

Confirmed via local Semgrep run BEFORE the fix:

  ❯❯❱ python.lang.security.audit.formatted-sql-query.formatted-sql-query
       ❰❰ Blocking ❱❱
        140┆ await db.execute(vacuum_sql)
  ❯❯❱ python.sqlalchemy.security.sqlalchemy-execute-raw-query.sqlalchemy-execute-raw-query
       ❰❰ Blocking ❱❱
        140┆ await db.execute(vacuum_sql)

The f-string lived on line 139 (the assignment), but Semgrep's taint
analysis reported the match on line 140 (the call site where
vacuum_sql is consumed). Annotations on lines 137-138 were too far.

Fix: collapse the f-string back inline into the db.execute() call,
then put a single combined # nosemgrep on the line directly above:

    # nosemgrep: formatted-sql-query, sqlalchemy-execute-raw-query
    await db.execute(f"VACUUM INTO '{backup_path}'")  # noqa: S608

Also added a code comment explaining the immediate-line constraint
so a future contributor doesn't re-split this and re-trigger the
alert.

Local Semgrep AFTER the fix: 0 findings, 0 blocking.

The same finding class re-fired on PR #127 twice — first cleared by
M7's annotation, then re-triggered after a refactor moved the f-string
to a variable. The inline form is more durable: refactors that keep
the call shape intact won't re-flag.

Refs: PR #127 — Semgrep alerts #82, #85, #86 (post-M13 scan); same
rules originally addressed in M7.

* fix(cli,tests): two CodeRabbit findings on M14 (M15)

1. providers.py: register_provider used `url in bundled` (raw) but
   bundled keys are normalised. User passing trailing-slash URL
   bypassed the bundled-collision check and silently overrode the
   bundled entry on next load_registry. Fix: lookup_by_url + scope
   refusal to bundled names only.

2. test_docker_e2e.py: tightened test_wrap_injects_base_url assertion
   from `'127.0.0.1' in base_url` to exact-match on the alias-qualified
   URL so a regression dropping /{alias}/v1 surfaces here.

Refs: PR #127 — CodeRabbit threads on M14.

* fix(tests): bump install-docker subprocess timeouts (M16)

CI runner load on GitHub Actions has grown; 240s docker-build /
360s docker-compose timeouts are now too tight for the install.sh
host-matrix workflow on PR #127. Same workflow passed 5 days ago
on main; nothing in 8rqs's diff slows the build (we don't touch
Dockerfile.ubuntu-bare, install.sh fetches PyPI's published
worthless which we haven't changed).

Build does: apt-get update + ca-certs/curl/tar + install.sh
(downloads uv + worthless from PyPI). Compose adds: mock-upstream
build + lock_e2e.py runtime. Both are network-bound; runner load
makes them variable.

Bumped: 240s → 480s for build, 360s → 600s for compose. Outer job
timeout is 25min so 5x headroom remains. Comments document why.

This is a fixture timeout adjustment, not a 8rqs feature change.
The actual lock-lifecycle test that exercises 8rqs's flow continues
to use the older PyPI worthless via install.sh (same as before).

* fix: 2 more CodeRabbit findings (M17)

1. providers.py: _atomic_write_text + read_text without encoding=utf-8
   TOML spec mandates UTF-8 but Python defaults to platform locale.
   M6 fixed read_text in cli/providers.py (loader); this fixes the
   write side in commands/providers.py (register) and the read side
   for the existing-file append at line 219. Mirrors the same UTF-8
   contract.

2. schema.py: _migrate_decoy_hash_column never converges on rerun
   The early return on column-exists also skipped the CREATE INDEX,
   so a DB left mid-migration (ALTER succeeded, index creation crashed)
   stayed broken across subsequent migrate_db() calls. Now: ALTER is
   column-conditional, CREATE INDEX always runs (idempotent).

Both Minor severity, both real correctness gaps. No new tests — fix
is small and the existing migration suite exercises both paths.

Refs: PR #127 — CodeRabbit threads PRRT_kwDORnDwR85_k153 (UTF-8)
and PRRT_kwDORnDwR85_k159 (index convergence).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants