feat(consent): reload persisted ledgers after restart - #140
Conversation
A same-millisecond research revocation must remain the latest purpose decision after the runtime reloads PostgreSQL consent evidence. Missing participants stay absent, out-of-order stored history fails closed, and stronger isolation is rejected. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Reconstruct consent events by occurrence time, then insertion time, then event identity so a later-inserted same-millisecond revoke is not hidden behind a lexicographically later grant after process restart. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Traceability, ERD, UML, as-built schema, ADR-0015, and doctoring now distinguish persist-on-main from this reload slice and cite PostgreSQL READ COMMITTED isolation. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
There was a problem hiding this comment.
Stale comment
Review — consent ledger reload
Unique slice is sound:
load_consent_ledgerreconstructs one participant ledger from existingconsent_ledger/consent_eventunderREAD COMMITTED, returnsNonefor a missing participant, and fails closed on unknown labels, non-monotonic stored time, stronger isolation, and missing relations. The same-millisecond research revoke RED is the buyer-visible case. This does not add HTTP or outbox composition. Do not fold #120/#123/#142 into this head.Residuals that should not block this slice, but must stay honest:
FOR SHAREon the header does not serialize withpersist_consent_ledger. Persist onlyINSERTs the header (ON CONFLICT DO NOTHING) and thenINSERTs events. A concurrent append is not locked out by the share lock. The rustdoc overclaims that the lock prevents a hidden in-flight append.- Third-order
event_ref ASCis the opaque-key tie-break #134/#142 reject for the outbox tail. Reload should treat a(occurred_at_unix_ms, created_at)collision asCorruptHistoryrather than inventing lexicographic order.clock_timestamp()makes this rare; do not silently reorder grant/revoke if it happens.InvalidTimestampdisplay still says the value exceeds the bigint range, but reload also uses that variant for a stored negative time.Do not merge until independent last-push review and exact-head required checks pass. Do not self-approve. Prefer this head over a second consent-reload PR. Next after this lands: keep outbox on #120/#142, then
persist_consent_ledger_with_outboxas the sole write path.Sent by Cursor Automation: Fix Issues
There was a problem hiding this comment.
Do not merge f223e749. The reload slice is the right scope (no outbox, no HTTP, no new tables), but load_consent_ledger does not implement the fail-closed reconstruct the tests and docs claim.
Blocking reconstruct defect
ORDER BY occurred_at_unix_ms ASC, created_at ASC, event_ref ASC then ConsentLedger::record reorders the corrupt-history fixture. The test inserts grant@20000 first and revoke@19000 second. After the ORDER BY, reconstruct is revoke@19000 then grant@20000. record() allows a revoke as the first event and only rejects time that moves backwards, so both appends succeed and the latest service_operation decision is granted. non_monotonic_stored_events_fail_closed_instead_of_reordering therefore cannot pass against this head; rustdoc that says out-of-order history fails closed instead of being reordered into a newer grant is false.
event_ref ASC is the same opaque tail #134 / #142 already reject. If two rows share (occurred_at_unix_ms, created_at), identity order can put consent_event_aaa_reload_revoke before consent_event_zzz_reload_grant and leave research granted after a revoke. Equal created_at must be CorruptHistory.
Required repair (keep this the only reload vehicle)
- Reconstruct in physical insertion order (
created_at ASC). Let domain monotonicity turn an earlieroccurred_atafter a later one intoCorruptHistory. - Add a RED that forces the same
created_aton azzzgrant andaaarevoke at the sameoccurred_at. Required result:CorruptHistory, not identity-ordered grant-latest. - Stop documenting “then event identity” in TRACEABILITY / ERD / AS_BUILT / ADR-0015 / rustdoc.
- Either take a persist-side header lock that
FOR SHAREcan wait on, or stop claiming the share lock hides in-flight appends.persist_consent_ledgercurrently onlyINSERTs the header.
Unknown labels, isolation, blank aliases, missing relations, and absent-vs-empty are real and correctly fail closed. No new PII masking. No psychometric kernel recreation.
I will land the reconstruct repair on a successor from this exact slice rather than folding outbox/HTTP into #140. Prefer that successor over this head. Keep outbox on #142. Do not self-approve. Independent last-push review and exact-head required checks still gate merge.
Sent by Cursor Automation: Fix Issues
| consent_form_version_ref, research_scope_ref, occurred_at_unix_ms \ | ||
| FROM consent_event \ | ||
| WHERE participant_ref = $1 \ | ||
| ORDER BY occurred_at_unix_ms ASC, created_at ASC, event_ref ASC", |
There was a problem hiding this comment.
ORDER BY occurred_at_unix_ms ASC then record() reorders the corrupt-history fixture (grant@20000 inserted first, revoke@19000 second) into revoke-then-grant. Domain allows a revoke as the first event, so the latest decision becomes a grant. event_ref ASC is also the opaque tail #134/#142 fail closed on. Reconstruct by created_at (insertion), fail closed on a created_at collision, and let NonMonotonicTimestamp become CorruptHistory.
| let mut transaction = client.transaction().unwrap(); | ||
| assert!( | ||
| matches!( | ||
| load_consent_ledger(&mut transaction, "participant_consent_reload_corrupt"), |
There was a problem hiding this comment.
This CorruptHistory assertion cannot pass against the current loader. After the ORDER BY, record(revoke@19000) on an empty ledger succeeds and record(grant@20000) succeeds. Either the reconstruct path is wrong, or this test is theater. Add a RED that forces equal created_at on zzz grant / aaa revoke as well.
| .is_none() | ||
| { | ||
| return Ok(None); | ||
| } |
There was a problem hiding this comment.
FOR SHARE only waits for writers that lock consent_ledger. persist_consent_ledger never takes FOR UPDATE on that row. A concurrent committed append can still appear between this lock and the event SELECT. Take a matching persist-side header lock or stop claiming the share lock hides in-flight appends.
| | Optional Keyverse account linking | PRD §3.1, §9.7 | TRD §10; UML identity-link lifecycle | ADR-0003, ADR-0020 | **Partially implemented**: issuer-scoped first-link fail-closed domain primitive in `src/participant.rs`; append-only unlink/relink/recovery history, persistence, audit, and transport remain Target | | ||
| | Cross-cutting tenant/task authorization | PRD §7, §9 | TRD §11; Security/Data | ADR-0001, ADR-0003 | **Implemented** fail-closed domain gate in `src/authorization.rs` binds consent operations to participant-owned `ConsentLedger` / `ManageOwnConsent`; persistence/policy-adapter/public-transport integration remains Target | | ||
| | Purpose-specific consent | PRD §5, §9.6 | TRD §12 | ADR-0006 | **Implemented** domain contract in `src/consent.rs` plus `migrations/0005_consent_lifecycle.sql` / `src/postgres_consent.rs` purpose-specific ledgers; HTTP transport remains Target | | ||
| | Purpose-specific consent | PRD §5, §9.6 | TRD §12 | ADR-0006 | **Implemented** domain contract in `src/consent.rs` plus `migrations/0005_consent_lifecycle.sql` / `src/postgres_consent.rs` purpose-specific ledgers; HTTP transport remains Target. Active PR #140 reloads a persisted ledger after restart, reconstructing events by occurrence time, then `created_at`, then event identity, so a same-millisecond research revocation remains the latest decision | |
There was a problem hiding this comment.
“then event identity” is the rejected #134 rustdoc. Reload may use created_at to keep insertion order when occurred_at ties. It must not break a created_at collision by event_ref. Same wording is copied into ERD / AS_BUILT / ADR-0015.


Why
Protected main can persist purpose-specific consent, but it cannot reconstruct that ledger after process restart. A buyer who grants research contribution and later revokes it in the same millisecond would appear granted again if operators or a later HTTP consent path rebuilt state only from memory.
This work is superseded by #156. Exact ancestry was reverified before closure: #156 is one commit ahead of this unchanged head and carries the full reload slice plus the fail-closed physical insertion-order/collision repair. Do not merge this predecessor.
Supersession
fix(consent): fail closed on reordered stored history)f223e749266eccda8964681cee5b7336231056e67fa917ca3ebe9c296d196fd720db54298f9941c7aheadby exactly 1 commit, behind by 0.The successor remains subject to unchanged exact-head CI/security/review gates. Closing this PR does not promote the successor to protected-main truth.