docs: add k6 preparation steps to the dev-database fixture runbook

docs/runbooks/simulation-fixture-on-a-dev-database.md

@@ -9,10 +9,12 @@ production history, in the database your IDE-run API is already pointed at.
 
 - [`tools/simulation/README.md`](../../tools/simulation/README.md) — the #243
   load-test harness. Same fixture, but in its own throwaway `cluckwork-sim`
-  compose stack on `:8081`, driven by `reset.sh`. If you want k6, Playwright or
-  a findings doc, go there and **do not** use this runbook. `reset.sh` and
-  `run-baseline.sh` `down -v` their own stack and must never be aimed at a
-  debug database.
+  compose stack on `:8081`, driven by `reset.sh`. If you want a findings doc,
+  the `docker stats`/Postgres samplers or a Playwright canary, go there.
+  `reset.sh` and `run-baseline.sh` `down -v` their own stack and must never be
+  aimed at a debug database. Pointing `k6 run` itself at the database this
+  runbook seeds *is* supported — see
+  [Preparing this database for k6](#preparing-this-database-for-k6).
 - [`first-admin-provisioning.md`](first-admin-provisioning.md) — the database
   has no Owner. Do that first; this runbook needs one.
 - `seed --profile demo` ([280](../decisions/280-seed-and-simulation.md)) — the
@@ -212,7 +214,7 @@ as environment variables the separator is `__`.
 | Key | Default | What it does |
 |---|---|---|
 | `Simulation__CastPassword` | *(none — required)* | Shared password for every `sim-*` cast member at whatever `Simulation__EmailDomain` is set to. Choose one at run time; never commit it. |
-| `Simulation__HistoryDays` | `90` | Depth of production history. See the table and the ceiling above. |
+| `Simulation__HistoryDays` | `90` | Depth of production history. See the table above; there is no ceiling, only the build cost. |
 | `Simulation__Managers` | `1` | Cast size, on top of the existing Owner. Managers place flocks and create products/categories/expenses. |
 | `Simulation__Sales` | `1` | Books the orders. |
 | `Simulation__Workers` | `3` | Records the daily entries. One is deliberately restricted to a single flock (#500). |
@@ -227,8 +229,11 @@ cast validates. Only `HistoryDays` 12 and 90 are covered by tests, though —
 treat anything far from those as untested, not as broken.
 
 **The cast counts must match `tools/simulation/.sim-cast.json` (git-ignored; generated by [`bootstrap.sh`](../../tools/simulation/bootstrap.sh)) if
-you later point k6 or Playwright at this database.** For hand debugging they can
-be anything.
+you later point k6 or Playwright at this database** — and so must
+`Simulation__CastPassword` and `Simulation__EmailDomain`. Take all of them from
+`.env.sim` rather than choosing them; the full procedure is
+[Preparing this database for k6](#preparing-this-database-for-k6). For hand
+debugging they can be anything.
 
 ## Verify
 
@@ -260,6 +265,224 @@ Not "it exited 0" — log in and look:
 | Exit `0`, but the screens you are debugging are still empty | Form A while an AppHost is up: it seeded the Compose database instead. Use form B. |
 | The web server starts instead of seeding | The binary was repeated in front of the verb, or `--` was omitted. |
 
+## Preparing this database for k6
+
+The [`tools/simulation/k6/`](../../tools/simulation/k6/) scripts normally run
+against the throwaway `cluckwork-sim` stack on `:8081`. They can run against
+the database this runbook just seeded instead: every script routes its URLs
+through one `BASE_URL`
+([`k6/config.js`](../../tools/simulation/k6/config.js)), so re-aiming them is
+one environment variable. What is *not* one environment variable is the
+fixture underneath — this section is the rest of it.
+
+> **`reset.sh` and `run-baseline.sh` are still off-limits.** Both `down -v`
+> the `cluckwork-sim` project — `run-baseline.sh` calls `reset.sh` once per
+> rep — and neither has any idea your debug database exists. Against a dev
+> database you invoke `k6 run` directly, and you get k6's own numbers only:
+> no `docker stats` sampler, no `pg_stat_statements` snapshot, no findings
+> doc. For any of those, use the harness stack as designed.
+
+**Blast radius, on top of the seed's own.** The personas write: Manager,
+Sales and Worker create daily entries, draft orders and expenses. Those rows
+push the account past the exact counts
+[step 1](#1-confirm-the-target-database-is-clean-and-is-the-one-you-mean)
+validates, so once k6 has run, the next `seed --profile simulation` against
+that database fails closed. That is recoverable only by a wipe — the same
+trap, reached from the other direction.
+
+### What has to line up
+
+k6 logs in as the ten users in `tools/simulation/.sim-cast.json`: the Owner
+`admin@<EmailDomain>` plus the nine cast members. `CAPACITY_VUS` defaults to
+10 and assigns one VU per user, so a missing or unusable Owner is a failed
+run, not a degraded one.
+
+| Requirement | Where it comes from |
+|---|---|
+| Cast emails and their shared password | `Simulation__CastPassword`, `Simulation__EmailDomain` and the four cast counts must be **the values `bootstrap.sh` generated**, not the hand-chosen password [step 3a](#3a-form-a--compose-dev-database-api-run-from-the-ide--cli) tells you to pick |
+| An Owner at `admin@<EmailDomain>`, usable | `bootstrap-admin` at that exact address, **then rotated off `MustChangePassword`** — [step k2](#k2-owner-create-and-rotate) |
+| `farmCode: "default-farm"` | Hardcoded in [`k6/auth.js`](../../tools/simulation/k6/auth.js). It is the default account's slug, set by the `AddAccountSlug` migration, so any base-seeded database already matches — nothing to do |
+| Rate limits raised past production values | [step k4](#k4-raise-the-rate-limits) |
+
+### k1. Generate the cast — `bootstrap.sh`
+
+Safe to run from here: it starts no container and names no compose project.
+It writes exactly two git-ignored files —
+`tools/simulation/.env.sim` and `tools/simulation/.sim-cast.json` — and exits
+early if `.env.sim` already exists.
+
+```bash
+bash tools/simulation/bootstrap.sh          # no-op if .env.sim is already there
+bash tools/simulation/bootstrap.sh --force  # new keypair + new passwords
+```
+
+`--force` invalidates the credentials any *existing* fixture was seeded with,
+in this database and in the sim stack. Both then need a wipe and a reseed.
+
+**Take only the `Simulation__*` and `SIM_ADMIN_*` lines out of `.env.sim`.**
+The rest of that file configures the sim stack's own containers — its
+Postgres credentials, its JWT keypair, `AllowedHosts=cluckwork-sim.local`,
+`ASPNETCORE_URLS=http://+:8080` — and feeding those to a dev API breaks it.
+The two `SIM_ADMIN_*` keys deliberately carry no `__`, so they are script
+values rather than app config (see `bootstrap.sh`'s own comment); this
+runbook reads them the same way `reset.sh` does.
+
+```bash
+env_val() { sed -n "s/^$1=//p" tools/simulation/.env.sim; }
+```
+
+### k2. Owner: create and rotate
+
+**This runs before the seed, not after it.** `seed --profile simulation`
+refuses to write a row unless the default account already holds an Owner
+(#500), and `bootstrap-admin` creates only the *first* one — against an
+account that already has an Owner it is a silent no-op (#283), so an Owner
+minted at any other address permanently blocks the one k6 needs. Create it
+at the address `.sim-cast.json` carries, on a database whose default account
+has no Owner yet ([step 1](#1-confirm-the-target-database-is-clean-and-is-the-one-you-mean)
+leaves it that way). This is the order `reset.sh` uses.
+
+Then rotate off the printed one-time password — until you do,
+`MustChangePasswordMiddleware` 403s every request the Owner VU makes (#283),
+and the run fails with the fixture looking fine.
+
+```bash
+ASPNETCORE_ENVIRONMENT=Development \
+  dotnet run --project src/Cluckwork.Api -- bootstrap-admin --email "$(env_val SIM_ADMIN_EMAIL)"
+```
+
+Under Aspire, prefix that with the same explicit `ConnectionStrings__Default`
+as above — it is a one-shot verb with exactly the #565 problem.
+
+`bootstrap-admin` prints `Temporary password: <value>` on stdout by design.
+**Do not echo it anywhere it will be retained** — a shell history or a CI log
+outlives the rotation (PR #392 review).
+
+Then, with the API *serving* against the same database, log in with the
+temporary password and change it to `SIM_ADMIN_PASSWORD`. That is exactly
+what `reset.sh` does after its own `bootstrap-admin` call — reuse the Python
+block there (`login` then `change-password`, `farmCode: "default-farm"`,
+`Idempotency-Key` on the write) rather than reimplementing it, pointing
+`APP_PORT` at whatever your dev API is listening on. Under Aspire the API
+`aspire run` launched is already serving against the right database; for the
+Compose form, start it yourself with an explicit `ASPNETCORE_URLS`.
+
+### k3. Seed with those values
+
+Run [step 3a](#3a-form-a--compose-dev-database-api-run-from-the-ide--cli) or
+[3b](#3b-form-b--aspire-apphost-stack) unchanged, except that every
+`Simulation__*` value comes from `.env.sim` instead of being chosen:
+
+```bash
+ASPNETCORE_ENVIRONMENT=Development \
+Simulation__CastPassword="$(env_val Simulation__CastPassword)" \
+Simulation__EmailDomain="$(env_val Simulation__EmailDomain)" \
+Simulation__Managers="$(env_val Simulation__Managers)" \
+Simulation__Sales="$(env_val Simulation__Sales)" \
+Simulation__Workers="$(env_val Simulation__Workers)" \
+Simulation__ReadOnly="$(env_val Simulation__ReadOnly)" \
+Simulation__HistoryDays="$(env_val Simulation__HistoryDays)" \
+Simulation__TimeZoneId="$(env_val Simulation__TimeZoneId)" \
+Simulation__Seed="$(env_val Simulation__Seed)" \
+  dotnet run --project src/Cluckwork.Api -- seed --profile simulation
+```
+
+For the Aspire stack, add form B's explicit
+`ConnectionStrings__Default=...` to that same command — a one-shot verb
+resolves the API's own user-secrets otherwise and seeds the *Compose*
+database (#565).
+
+The cast counts in `.env.sim` are `bootstrap.sh`'s copy of
+`SimulationOptions`' defaults, so passing them explicitly is belt and
+braces — but `Simulation__CastPassword` genuinely differs, and it is the one
+that decides whether k6 can log in.
+
+### k4. Raise the rate limits
+
+Production defaults are `Login` 10 per 900s, `Refresh` 60 per 900s,
+`ClientErrors` 10 per 300s
+([`RateLimitingOptions.cs`](../../src/Cluckwork.Api/RateLimiting/RateLimitingOptions.cs)).
+A baseline rep logs in 12 times before it starts, so it hits the login wall
+during warmup and the run is measuring the limiter. `.env.sim` raises all
+three to 1,000,000 for exactly this reason.
+
+Set them where **both** launch paths pick them up — the API's user-secrets.
+The AppHost injects only the connection string and the Redis key (#565), so
+these still bind under `aspire run`:
+
+```bash
+dotnet user-secrets --project src/Cluckwork.Api set "RateLimiting:Login:PermitLimit" "1000000"
+dotnet user-secrets --project src/Cluckwork.Api set "RateLimiting:Refresh:PermitLimit" "1000000"
+dotnet user-secrets --project src/Cluckwork.Api set "RateLimiting:ClientErrors:PermitLimit" "1000000"
+```
+
+Restart the API afterwards, and **remove them when you are done** — a debug
+box with no login limiter is not what you want to keep testing against:
+
+```bash
+dotnet user-secrets --project src/Cluckwork.Api remove "RateLimiting:Login:PermitLimit"
+dotnet user-secrets --project src/Cluckwork.Api remove "RateLimiting:Refresh:PermitLimit"
+dotnet user-secrets --project src/Cluckwork.Api remove "RateLimiting:ClientErrors:PermitLimit"
+```
+
+**Then restart the API again — removing the keys is not enough.** The
+section is bound once, at service registration, and the numbers are baked
+into the policy objects right there
+([`CluckworkRateLimitingServiceCollectionExtensions.cs`](../../src/Cluckwork.Api/Hosting/CluckworkRateLimitingServiceCollectionExtensions.cs)
+ — `Get<RateLimitingOptions>()`, then `new DistributedIpFixedWindowPolicy(…,
+rateLimiting.Login.PermitLimit, …)`). Nothing re-reads them, so a process
+left running keeps serving the 1,000,000 limits off a user-secrets file
+that no longer mentions them — the worst version of this, because the
+configuration now looks clean.
+
+### k5. Run k6
+
+`BASE_URL` is the only thing that moves. Start with the smoke script, which
+fails loudly on anything the fixture got wrong, before spending a baseline
+on it:
+
+```bash
+export BASE_URL=http://127.0.0.1:8080     # Aspire's LocalPorts:Api default
+
+k6 run tools/simulation/k6/auth-smoke.js
+k6 run --vus 5 --duration 20s tools/simulation/k6/persona-smoke.js
+CAPACITY_DURATION=2m k6 run tools/simulation/k6/baseline.js
+```
+
+`8080` is the Aspire `api` resource's committed `LocalPorts:Api` default, and
+it is overridable per machine — take the port the run actually used from the
+dashboard ([Aspire runbook](aspire-local-development.md)). For the Compose
+form it is whatever `ASPNETCORE_URLS` you started the API with. Either way it
+is the **API**, not Vite on `5173`: the SPA dev server proxies `/api` but
+serves none of the endpoints k6 measures.
+
+### Known gap: the `staticAssets` flow 404s
+
+`app.MapFallbackToFile("index.html", …)` is a **no-op in dev** — there is no
+`wwwroot`, because a dev SPA is served by Vite
+([`Program.cs`](../../src/Cluckwork.Api/Program.cs), the comment above the
+fallback). k6's `staticAssets` flow fetches `/` and a hashed asset off it, so
+against a dev-run API it fails two checks per iteration and bumps
+`unexpected_status` — a signal `persona-smoke.js` asserts must be zero.
+
+Two honest options, and neither is "ignore it":
+
+- **Read the numbers per flow.** Every metric is tagged `flow:staticAssets`,
+  so the API percentiles are still usable with that flow excluded. The
+  smoke scripts' own pass/fail is not — treat them as red for a reason.
+- **Give the API a `wwwroot`,** which is what the container does
+  ([`Dockerfile`](../../src/Cluckwork.Api/Dockerfile) copies `web/dist` into
+  it):
+
+  ```bash
+  (cd web && npm run build)
+  mkdir -p src/Cluckwork.Api/wwwroot && cp -r web/dist/. src/Cluckwork.Api/wwwroot/
+  ```
+
+  `src/Cluckwork.Api/wwwroot/` is **not** git-ignored and does not exist on a
+  clean checkout, so delete it when you are finished rather than leaving a
+  built SPA loose in the working tree.
+
 ## Drill
 
 Safe on a scratch database only.
@@ -280,3 +503,36 @@ Safe on a scratch database only.
    before #638.
 6. Sign in and walk the four **Verify** checks.
 7. Update **Last drilled** above.
+
+The k6 preparation path is a second drill, on its own scratch database — it
+needs the Owner at a *specific* address, so it cannot share step 2's admin:
+
+1. `docker compose -f deploy/docker-compose.dev.yml down -v && docker compose -f deploy/docker-compose.dev.yml up -d`,
+   so the default account has **no** Owner.
+2. [k1](#k1-generate-the-cast--bootstrapsh) — `bash tools/simulation/bootstrap.sh --force`,
+   and define `env_val`. Then [k2](#k2-owner-create-and-rotate):
+   `bootstrap-admin --email "$(env_val SIM_ADMIN_EMAIL)"` and rotate. Expected:
+   a `Temporary password:` line, **not** `Admin already provisioned` — that
+   message means the database was not clean and the rest of this drill is void.
+3. [k3](#k3-seed-with-those-values). Expected: exit `0`.
+4. [k4](#k4-raise-the-rate-limits), then restart the API.
+5. `BASE_URL=http://127.0.0.1:8080 k6 run tools/simulation/k6/auth-smoke.js`,
+   with `BASE_URL` pointing at **your dev API**. Set it explicitly: it
+   defaults to `http://127.0.0.1:8081`
+   ([`config.js`](../../tools/simulation/k6/config.js)), which is the sim
+   stack, so an unset `BASE_URL` drills a database this runbook never
+   touched. Expected: pass. It logs in as the Owner and all nine cast
+   members, so it is the check that k2's address and k3's `CastPassword`
+   actually agree.
+6. Run the k4 `remove` commands and restart the API again. Expected: the
+   same `BASE_URL=… k6 run …auth-smoke.js` now **fails** on 429s — `setup()` calls
+   `preflightCredentials` for all ten, then each VU logs in again, so one
+   run is ~20 logins from one IP against a restored `Login` budget of 10
+   per 900s. A pass here means the overrides are still live and you have
+   just proved the cleanup does not work.
+7. Update **Last drilled** above.
+
+Under Aspire, only two lines of that change: steps 2 and 3 are one-shot
+verbs and need form B's explicit `ConnectionStrings__Default`, or they
+drill the Compose database while you watch the Aspire one (#565). Step 6's
+expectation is the same either way.