feat(api): v1 shared primitives — error envelope, cursor codec, cost governor, auth tiers (G2) - #168
Conversation
The first of the four G2 shared primitives every /api/v1/* endpoint
inherits. A nested, machine-first error envelope
{error:{reason,message,detail}} (design v1-api-design.md §1.3) replacing
the frozen flat {reason,detail} compat shape, plus the canonical Reason
enum (§1.4 / coherence Part A) with a pure reason->HTTP-status mapping.
One enum, one spelling per concept, so a disabled subsystem answers its
*_disabled reason instead of a bare 404. 111 variants (110 canonical +
invalid_cursor, required by the §1.5 cursor codec). Status overrides off
the mechanical suffix rule are documented at each arm (unsupported_intent
->422, compiler/oracle_unavailable->501, timeout->504) and pinned by an
oracle-parity table test against the spec.
Not mounted on any route; the first route-group PR consumes it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BUh2DBnAPqThdYFZW5D8wx
The one cursor-pagination codec (design §1.5, coherence C.4): every v1 collection is cursor-paginated (?limit=&cursor=, never offset on the wire). Wire form is base64url-nopad(version-byte || compact-JSON payload), generic over a small CursorPayload each group picks (height key, global index, mempool keyset, ...). Opaqueness is load-bearing (locked decision D5): because clients treat the cursor as opaque, Phase-2 can swap a Phase-1 offset-alias payload for a stable-seek key without breaking clients; the version byte makes an incompatible future encoding fail closed (invalid_cursor) rather than mis-seek. Includes clamp_limit (per-group default/max from §2.2) and Page::from_overfetch (overfetch-by-one has_more, no total needed). Adds base64 (0.22, already in Cargo.lock via ergo-compiler) to ergo-api — no new crate. Tamper/version/shape rejections tested. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BUh2DBnAPqThdYFZW5D8wx
The load-bearing control behind the T0 'public but bounded' contract (design §2.1–§2.2): every open surface sits behind a per-IP token bucket whose refill/burst are config knobs, and each RouteClass (cheap read / heavy read / compute) consumes a class-weighted number of tokens. A depleted bucket answers 429 rate_limited (the §1.3 envelope) with a Retry-After header; the operator's own UI (loopback) is exempt by default. Dependency-light: a std HashMap<IpAddr,Bucket> behind a std Mutex, refilled lazily on access (no timer task), with opportunistic pruning of idle full buckets. Client IP comes from ConnectInfo (added shared client_ip helper; XFF deliberately untrusted); a connect-info gap falls back to one shared 'unknown' bucket, never an exemption. Attaches per subtree via from_fn_with_state(gov.state(class), governor_mw). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BUh2DBnAPqThdYFZW5D8wx
The exposure-tier gate (design §2.1): Public (no gate — governor-bounded), Operator (valid api_key), Admin (api_key AND loopback-preferred). Reuses the EXISTING api-key scheme — factored the Blake2b-256 + constant-time compare into ApiSecurity::verify and pointed the frozen compat gate (require_api_key) at it too, so there is one credential path node-wide. The v1 gate answers the §1.3 envelope (unauthorized/401) rather than the legacy 403 shape. Admin is warn-and-allow off-loopback by default (logged loudly), with a config flag to hard-deny remote admin ops (sensitive_op_disabled). Adds the startup boot-warn: assess_posture (pure, tested predicate) + warn_startup_posture, which screams when T1/T2 are network-reachable under no key or a known weak/default key (the shipped 'hello' template value). Not wired into server startup here (this PR changes no server behavior) — exported with its call site documented for the next PR. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BUh2DBnAPqThdYFZW5D8wx
📝 WalkthroughWalkthroughThis PR centralizes API-key verification into ChangesV1 API infrastructure
Estimated code review effort: 4 (Complex) | ~75 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
SECURITY: - auth.rs: require_tier fails CLOSED on NoKeyConfigured for Operator/Admin (401 unauthorized), Admin denies before the loopback check. Public unaffected. Matches the compat require_api_key credential-reject posture; an always-mounted admin gate (shutdown/secret-export) must DENY when no key is configured. The boot-warn stays a signal, not the gate. - mod.rs: is_trusted_loopback() is the single loopback-privilege authority; new local_reverse_proxy flag (default false) on GovernorConfig + V1AuthConfig withdraws socket-derived loopback trust (governor exemption AND Admin loopback-preferred) when an operator declares a local reverse proxy — no longer inferable from a proxy's socket. XFF not consulted. CORRECTNESS: - governor.rs: Governor::new -> Result<_, GovernorConfigError>; GovernorConfig:: validate rejects non-finite/non-positive refill+burst and non-finite/negative weights before any bucket math. - governor.rs: prune recomputes each bucket's virtual refill before retain, so idle-drained buckets that virtually refilled to full actually drop (max_tracked_ips is now an effective bound). - error.rs: Reason::RouteDisabled -> RouteUnavailable (wire route_unavailable, 503) — resolves the _disabled->409 vs _unavailable->503 suffix-rule inconsistency; 'route/bridge not wired' is transient = _unavailable. parse_id32 lowercase-only SKIPPED (superseded by #171 case-insensitive; also doesn't exist on this base branch). +14 unit tests (auth fail-closed matrix, governor validation/pruning/proxy, is_trusted_loopback). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BUh2DBnAPqThdYFZW5D8wx
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ergo-api/src/v1/governor.rs`:
- Around line 240-254: The prune logic in the governor’s `map.retain` block only
removes idle, fully refilled buckets, so `max_tracked_ips` can still be exceeded
indefinitely under many active peers. Update the eviction behavior in
`governor.rs` so the `Governor` table is actually bounded—either evict
additional entries when `map.len()` is over the limit or throttle the prune path
instead of scanning the full map on every request. Keep the existing refill/idle
checks in mind, but ensure the path around `idle_prune_after`,
`max_tracked_ips`, and `retain` can’t leave the map above the configured cap
forever.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 8ff85dbe-9751-4a7b-9c00-1e4012bf6a82
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (8)
ergo-api/Cargo.tomlergo-api/src/auth.rsergo-api/src/lib.rsergo-api/src/v1/auth.rsergo-api/src/v1/cursor.rsergo-api/src/v1/error.rsergo-api/src/v1/governor.rsergo-api/src/v1/mod.rs
| if map.len() > self.config.max_tracked_ips { | ||
| let cutoff = self.config.idle_prune_after; | ||
| // Recompute each bucket's virtual refill (same logic as the live | ||
| // charge above) BEFORE the predicate: an idle bucket's stored | ||
| // `tokens` is stale, so without this an idle-but-drained bucket | ||
| // looks non-full forever and `max_tracked_ips` never bounds the | ||
| // table. Drop a bucket only once it is idle past the cutoff AND | ||
| // has virtually refilled to full (a fresh entry also starts full, | ||
| // so nothing is lost). | ||
| map.retain(|_, b| { | ||
| let idle = now.saturating_duration_since(b.last); | ||
| let refilled = (b.tokens + idle.as_secs_f64() * refill).min(burst); | ||
| refilled < burst || idle < cutoff | ||
| }); | ||
| } |
There was a problem hiding this comment.
🚀 Performance & Scalability | 🟠 Major | 🏗️ Heavy lift
max_tracked_ips is only a soft cap here.
retain only evicts buckets that are both full and idle past the cutoff, so a workload with many active peers can keep the map above the configured limit indefinitely. Once that happens, every request pays an O(n) prune scan while holding the global Mutex, which hurts throughput under the exact high-cardinality load this is meant to absorb. Consider an eviction path that can actually bound the table, or rate-limit the prune scan.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ergo-api/src/v1/governor.rs` around lines 240 - 254, The prune logic in the
governor’s `map.retain` block only removes idle, fully refilled buckets, so
`max_tracked_ips` can still be exceeded indefinitely under many active peers.
Update the eviction behavior in `governor.rs` so the `Governor` table is
actually bounded—either evict additional entries when `map.len()` is over the
limit or throttle the prune path instead of scanning the full map on every
request. Keep the existing refill/idle checks in mind, but ensure the path
around `idle_prune_after`, `max_tracked_ips`, and `retain` can’t leave the map
above the configured cap forever.
…floods (#186) The per-IP token-bucket prune in `charge_at` only dropped buckets that were BOTH idle past the cutoff AND virtually refilled to full. Under a flood of many distinct ACTIVE peers (all `idle < cutoff` — trivial over IPv6) nothing is pruned, so `max_tracked_ips` never bounds the map: it grows with the attacker's IP count, a memory-exhaustion DoS. `max_tracked_ips` was documented as a cap but was not enforced as one (CodeRabbit, #168 review). Add a second, hard-bound pass after the lossless idle prune: while the table is over `max_tracked_ips`, evict the LEAST-throttled bucket (highest virtual tokens). A near-full bucket sheds ~no enforcement when evicted (a fresh entry is recreated at `burst`), while a drained/throttled bucket keeps its penalty — so an attacker's throttled buckets are preserved and only the least-penalized IPs lose tracking. That trade (imperfect per-IP tracking for the least-penalized IPs) is correct against unbounded memory. Only a fresh insert can exceed the cap, so in steady state this evicts at most one bucket per charge; the existing idle prune is unchanged. Tests: active-peer flood of 50 distinct IPs stays within max_tracked_ips; hard eviction sheds the near-full bucket and keeps the throttled one; the two existing idle-prune tests still pass. Claude-Session: https://claude.ai/code/session_01BUh2DBnAPqThdYFZW5D8wx Co-authored-by: arkadianet <rkadias@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* wip: v1 OpenAPI registration — DTOs/error + chain/boxes/tokens/addresses/mempool Intermediate commit for a large in-progress change (will be squashed or left as history at PR time): ToSchema on every v1 DTO + the shared V1Error/Reason envelope, and typed #[utoipa::path] annotations for chain/*, boxes/*, tokens/*, addresses/*, mempool/* (still not registered into any OpenApi derive yet — that lands once every v1 handler is annotated). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018eXcaurqKT9o1ngi8cvjTC * wip: v1 OpenAPI registration — transactions/tx_intel/decode/light/stats/diagnostics/batch Continuing the annotation pass (still not registered into an OpenApi derive). transactions.rs, tx_intel.rs, decode.rs, light.rs, stats.rs, diagnostics.rs, batch.rs all now carry typed #[utoipa::path] annotations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018eXcaurqKT9o1ngi8cvjTC * wip: v1 OpenAPI registration — accounts/scan Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018eXcaurqKT9o1ngi8cvjTC * wip: v1 OpenAPI registration — operator (node/network/mining/voting) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018eXcaurqKT9o1ngi8cvjTC * wip: v1 OpenAPI registration — script/webhooks/realtime (all v1 handlers now annotated) Every /api/v1/* handler across the v1 product surface now carries a #[utoipa::path] annotation. Next: register everything into a new V1OpenApi derive + mount /api-docs/openapi-v1.yaml + /swagger/v1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018eXcaurqKT9o1ngi8cvjTC * feat(api): register the whole v1 API surface in OpenAPI The v1 product API (~122 handlers across chain/boxes/tokens/addresses/ mempool/transactions/tx-intelligence/script/decode/light/stats/ diagnostics/webhooks/realtime/scan/accounts/operator, from the 13 merged PRs #168-186) had zero OpenAPI documentation: none of it was listed in NativeOpenApi's paths(), and there was no dashboard UI entry point for any v1 subsystem. Give v1 its own OpenAPI document (/swagger/v1, /api-docs/openapi-v1.yaml) rather than folding it into the pre-v1 NativeOpenApi — it's a distinct product surface with its own T0/T1/T2 auth tiers and cost governor, and /swagger + /swagger/native already coexist as separate specs. - ToSchema on every v1 DTO and the shared V1Error envelope - a real #[utoipa::path] per handler, matching the wallet/native house style (typed responses, per-status-code error variants keyed off Reason::http_status()) - widened v1 mod/handler/DTO visibility to pub(crate) so the new cross-cutting V1OpenApi derive can name them from outside their home modules (no new public API surface) - new ergo-api/src/v1/openapi.rs (V1OpenApi derive + V1SecurityAddon) mounted at /swagger/v1, /api-docs/openapi-v1.{yaml,json}, linked from the dashboard sidebar - golden-snapshot test (openapi_v1_snapshot.rs) mirroring the existing openapi_native_snapshot.rs convention Test plan: - cargo fmt --all -- --check - cargo clippy --workspace --all-targets --all-features -- -D warnings - cargo test --workspace (incl. new openapi_v1_matches_snapshot) - live smoke-test: built ergo-node from this branch, ran it against testnet on a throwaway port, confirmed /swagger/v1, /api-docs/openapi-v1.yaml, and /api-docs/openapi-v1.json all return 200 with the expected spec content, and that the dashboard sidebar carries the new "V1 API" link Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018eXcaurqKT9o1ngi8cvjTC * fix(api): document 7 missing response codes on v1 OpenAPI paths CodeRabbit review of PR #188 found several handlers whose #[utoipa::path] responses() didn't cover every reachable Reason: - mining::solution — missing 503 (map_mining_error can return CandidateUnavailable, same as candidate) - batch::batch_handler — missing 429 (governor.try_charge -> RateLimited) - decode::protocol_state, boxes::box_by_id, transactions::tx_by_id, addresses::transactions, tx_intel::status — missing 500 (InternalError from the shared box/tx response assembly helpers) - webhooks::patch_active — missing 400 (V1Json rejection -> BadRequest) All are documentation-only: the handlers already produced these responses, the OpenAPI spec just didn't say so. Regenerated the golden fixture (openapi_v1.yaml) to match. Test plan: - cargo fmt --all -- --check - cargo clippy -p ergo-api --all-targets --all-features -- -D warnings - cargo test -p ergo-api (incl. regenerated openapi_v1_matches_snapshot) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018eXcaurqKT9o1ngi8cvjTC * fix(ci): pin openapi_v1.yaml fixture to LF line endings Windows CI's openapi_v1_matches_snapshot failed: "line counts differ: expected 10013, actual 10013" — the byte-compare diverged on line endings, not content, because the fixture wasn't in .gitattributes' eol=lf pin (only openapi_native.yaml was). Windows checkout autocrlf rewrote it to CRLF while the generated spec is always LF. Same fix already applied to openapi_native.yaml when it hit the same failure; just extending the existing pin to the new v1 fixture. Test plan: - git add --renormalize confirms the committed blob is already LF-only (no working-tree diff), so this is a checkout-behavior fix only --------- Co-authored-by: arkadianet <rkadias@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Bump workspace version to 0.5.2 and promote the changelog: the complete v1 product API (#168-#185, #188), shadow validation as a production mode (#193-#195), the operator observability wave (#187, #190, #192, #194), two live accept-invalid consensus fixes (#176, #179), ErgoScript compiler byte-parity completion (#165-#167, #175), and the #160-#163 sync/recovery fixes. Full workspace gate run on the merge result. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BUh2DBnAPqThdYFZW5D8wx
What this is
The first PR of the native
/api/v1/*build-out — the four shared primitives every v1 endpoint will inherit (the design's "convention-lock" step). Pure infrastructure + tests: no routes are mounted, no existing behavior changes (compat surfaces untouched; the oneauth.rschange is a behavior-preserving refactor extracting averifymethod so v1 reuses the existing api_key path instead of inventing a second scheme).v1/error.rs— the nested error envelope{"error":{reason,message,detail}}+ the canonicalReasonenum (111 variants, lowercase_snake) with the full HTTP-status mapping. A subsystem-off endpoint returns its reason (indexer_disabled→ 409), never a bare 404. The enum is pinned by a table test asserting every variant's exact wire string + status.v1/cursor.rs— the one opaque, versioned, URL-safe cursor codec + the{limit, next_cursor, has_more}page builder with limit clamping. The encoding is opaque by design so Phase-2 can swap offset-alias payloads for stable seeks without breaking clients; tamper →invalid_cursor(400).v1/governor.rs— the per-IP token-bucket rate/cost governor (per-route-class weights: cheap read / heavy read / compute), 429 +Retry-Afterin the envelope, loopback exempt by default. This is the load-bearing control that makes T0-public surfaces safe.v1/auth.rs— the T0/T1/T2 tier split (Public / Operator api_key / Admin api_key+loopback-preferred) reusing the existing credential verification, pluswarn_startup_posture(the boot-warn for weak/default keys on network-reachable T1/T2) — exported with its call site documented; wired by the first route-group PR.Notes for review
-D warnings/ test).base64 0.22added to ergo-api — already resolved in the workspace lock (no new crate).into_make_service_with_connect_info::<SocketAddr>()(governor/auth read the client IP fromConnectInfo) — documented inv1/mod.rs.invalid_cursoradded to the enum (required by the cursor spec, omitted from the enum list);compiler_unavailable/oracle_unavailable→ 501 per the spec's own "built-without" note; no-key-configured on T1/T2 passes (parity with today's dev behavior — the boot-warn is the loud guard).Next PRs consume this: the read-gap groups (
chain/*,transactions/*reads,boxes/tokens/addresses).🤖 Generated with Claude Code
https://claude.ai/code/session_01BUh2DBnAPqThdYFZW5D8wx
Summary by CodeRabbit
New Features
Bug Fixes