Skip to content

feat(outbox): transactional outbox + relay (AddCaliberOutbox, RelayHost) (#36)#61

Merged
williamdewitt merged 1 commit into
mainfrom
feat/m36-outbox-relay
Jun 27, 2026
Merged

feat(outbox): transactional outbox + relay (AddCaliberOutbox, RelayHost) (#36)#61
williamdewitt merged 1 commit into
mainfrom
feat/m36-outbox-relay

Conversation

@williamdewitt

Copy link
Copy Markdown
Owner

Summary

Transactional-outbox mode — the heart of M2 — behind the documented one-liner:

builder.Services.AddCaliberWebhooks(o => o.UseEntityFramework<AppDbContext>());
// in AppDbContext:
protected override void OnModelCreating(ModelBuilder b) => b.AddCaliberOutbox();

await _webhooks.PublishAsync("order.shipped", new { orderId, trackingNo });
await db.SaveChangesAsync();   // ONE transaction: business data + outbox row
  • AddCaliberOutbox(ModelBuilder) — maps the thin caliber_outbox table onto the caller's model (they migrate it via their normal dotnet ef workflow). created_at is stored as UTC ticks so the relay's ORDER BY translates on SQLite (which cannot ORDER BY a DateTimeOffset) and Postgres alike.
  • OutboxPublisher<TContext> (scoped IWebhookPublisher) — PublishAsync stages a row into the caller's scoped DbContext and returns; no SaveChanges. The caller's SaveChangesAsync commits it atomically with their business data (no-magic-SaveChanges, design feat(config): AddCaliberWebhooks options + validation #6).
  • RelayProcessor + RelayHost<TContext> (BackgroundService) — drains committed outbox rows oldest-first, fans out to matching endpoints (as-of-relay-time), inserts via the existing IMessageStore.AddAsync (idempotent on (event_id, endpoint_id)), then deletes the drained rows. Crash-safety rests on that idempotency: a crash between insert and delete re-fans-out and no-ops on retry. Mirrors DispatcherHost's PeriodicTimer/TimeProvider drain-while-full + transient-survival idiom.
  • UseEntityFramework<TContext>() — wires the EF stores against TContext's own database (provider + connection read from the caller's context; SQLite in v1, Postgres lands with feat(storage): durable Postgres store + shipped migrations (FOR UPDATE SKIP LOCKED) #38), the scoped outbox publisher, the relay, and a shared-schema initializer that CREATE TABLEs Caliber's messages/endpoints into the caller DB when absent (idempotent; production Postgres ships reviewed migrations in feat(storage): durable Postgres store + shipped migrations (FOR UPDATE SKIP LOCKED) #38).
  • CoreIngestionManager publisher registration → TryAddSingleton, so a provider's StoreConfigurator can supply an outbox-mode publisher that wins.

Both standalone (direct PublishAsyncmessages) and outbox (stage → relay) modes work behind the same IWebhookPublisher; the guarantee tier surfaces at the store-config line, never a silent downgrade.

Design note (reviewers)

The design doc floated a single-transaction relay (insert messages + delete outbox in one tx). This PR implements the issue-#36 acceptance criteria literally — fan-out via the existing IMessageStore + idempotent insert — which gives the same crash-safety through idempotency rather than a shared transaction, and lets messages reuse the exact store the dispatcher already claims from. transactional-outbox.md is reconciled to match.

Closes

Closes #36. Part of the #8 M2 epic. (SQLite tier #37 already merged via #59/#60; Postgres + migrations is #38.)

Test plan

  • dotnet build -c Release — 0 warnings (net8 + net10)
  • dotnet test -c Release203/203 (+11: staging-doesn't-commit, fan-out single/multi, idempotent re-drain, unmatched drains, oldest-first, empty, null guards, DI wiring)
  • CI: build · test · pack · mutation diff-gate

🤖 Generated with Claude Code

…st) (#36)

Transactional-outbox mode behind the documented one-liner:

  builder.Services.AddCaliberWebhooks(o => o.UseEntityFramework<AppDbContext>());
  // in AppDbContext: protected override void OnModelCreating(ModelBuilder b) => b.AddCaliberOutbox();

- AddCaliberOutbox(ModelBuilder): maps the thin caliber_outbox table onto the caller's model (they own
  its migration); columns snake_case. created_at stored as UTC ticks so the relay's ORDER BY translates
  on SQLite (which can't ORDER BY a DateTimeOffset) and Postgres alike.
- OutboxPublisher<TContext> (scoped IWebhookPublisher): PublishAsync STAGES a row into the caller's
  scoped DbContext and returns — no SaveChanges. The caller's SaveChangesAsync commits it atomically
  with their business data. No-magic-SaveChanges per design #6.
- RelayProcessor + RelayHost<TContext> (BackgroundService): drains committed outbox rows oldest-first,
  fans out to matching endpoints (as-of-relay-time), inserts via the existing IMessageStore.AddAsync
  (idempotent on (event_id, endpoint_id)), then deletes the drained rows. Crash-safety rests on that
  idempotency — a crash between insert and delete re-fans-out and no-ops on retry. Mirrors DispatcherHost's
  PeriodicTimer/TimeProvider drain-while-full + transient-survival idiom.
- UseEntityFramework<TContext>(): wires the EF message/endpoint stores against TContext's own database
  (provider + connection read from the caller's context; SQLite in v1, Postgres arrives with #38),
  the scoped outbox publisher, the relay, and a shared-schema initializer that CREATE-TABLEs Caliber's
  messages/endpoints into the caller DB when absent (idempotent; production Postgres ships migrations in #38).
- Core: IngestionManager publisher registration → TryAddSingleton, so a provider's StoreConfigurator can
  supply an outbox-mode publisher that wins.

Both standalone (direct PublishAsync → messages) and outbox (stage → relay) modes work behind the same
IWebhookPublisher; the guarantee tier surfaces at the store-config line, never a silent downgrade.

11 new tests (203 total, 0 warnings, net8+net10): staging-doesn't-commit, payload/clock stamp, fan-out
(single + multi endpoint), idempotent re-drain (crash-before-delete), unmatched-event drains, oldest-first
ordering, empty-outbox, null guards, and UseEntityFramework DI wiring. Docs: transactional-outbox.md relay
section reconciled to the idempotency-based crash-safety the code actually implements.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions github-actions Bot added the risk:core src/Caliber.Webhooks; code-owner review (Opus, high effort) label Jun 27, 2026
@williamdewitt williamdewitt merged commit 703e42b into main Jun 27, 2026
9 checks passed
@williamdewitt williamdewitt deleted the feat/m36-outbox-relay branch June 27, 2026 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:core src/Caliber.Webhooks; code-owner review (Opus, high effort)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(outbox): transactional outbox + relay (AddCaliberOutbox, RelayHost)

1 participant