feat(storage): durable Postgres store + shipped migrations (FOR UPDATE SKIP LOCKED) (#38)#62
Merged
Merged
Conversation
…E SKIP LOCKED) The scale-up production tier (#38). `UsePostgres(conn)` (standalone) and the outbox `UseEntityFramework` path now target Npgsql, sharing one `CaliberNpgsqlConfiguration` so every entry point configures the provider identically. EfMessageStore's claim path is one atomic statement per provider: on Postgres `UPDATE … FROM (SELECT … FOR UPDATE SKIP LOCKED LIMIT @Batch) … RETURNING`, so N dispatchers claim disjoint batches with no contention and no double-send; the SQLite variant keeps the equivalent `UPDATE … RETURNING` (single writer serialises it). Fan-out stays idempotent via `ON CONFLICT (event_id, endpoint_id) DO NOTHING`. Schema is provisioned by a shipped, reviewed migration (InitialCreate), applied on startup by CaliberWebhooksPostgresMigrator (standalone) and the provider-aware CaliberWebhooksSharedSchemaInitializer (outbox). Caliber tracks its migrations in a dedicated `__caliber_migrations_history` table so they never collide with the caller's own migrations when sharing a database. Microsoft.EntityFrameworkCore.Design is design-time only (PrivateAssets=all). The cross-instance no-double-send proof (Testcontainers) is #39, blocked by this; its test-only deps land there. Closes #38 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #38 — the scale-up production storage tier for the M2 epic (#8).
risk:critical(ships EF migrations → migration-safety review; does not auto-merge).What this delivers
AddCaliberWebhooks(o => o.UsePostgres("Host=…;Database=…;Username=…;Password=…"))UseEntityFramework<TContext>()now detects an Npgsql caller context and shares its database.CaliberNpgsqlConfiguration.Apply(...)so every entry point (standalone, outbox, design-time) configures Npgsql identically.EfMessageStore:UPDATE messages … FROM (SELECT id … WHERE due ORDER BY next_attempt_at FOR UPDATE SKIP LOCKED LIMIT @batch) … RETURNING …— N dispatchers claim disjoint batches with no contention.UPDATE … RETURNING; the single writer serialises it.(event_id, endpoint_id)viaON CONFLICT DO NOTHING(both providers).InitialCreate) applied on startup —CaliberWebhooksPostgresMigrator(standalone) / provider-awareCaliberWebhooksSharedSchemaInitializer(outbox: migrate on PG, create-tables-if-missing on SQLite). NoEnsureCreatedin production.__caliber_migrations_historytable so they never collide with the caller's own migrations when Caliber shares the caller's database (outbox mode).Microsoft.EntityFrameworkCore.Designis design-time only (PrivateAssets=all) — never flows to consumers. Npgsql provider pinned to the EF Core 8 line (8.0.11) to unify with the existing Sqlite 8.0.28 pin under transitive pinning.Migration-safety review
Generated SQL (
dotnet ef migrations script --idempotent) is purely additive and idempotent —CREATE TABLE/CREATE INDEXguarded byIF NOT EXISTS … __caliber_migrations_history. NoDROP, no destructiveALTER, no data movement. Indexes match the access paths:IX_messages_status_next_attempt_at— the claim predicate +ORDER BY.IX_messages_event_id_endpoint_id— backs idempotent fan-out (ON CONFLICT).IX_endpoints_enabled— endpoint lookup.Verification
dotnet build— 0 warnings, 0 errors (net8.0 + net10.0).dotnet test— 203 / 203 pass on both targets.dotnet ef migrations scriptgenerates clean Postgres DDL.Scope / follow-ups
Testcontainers.PostgreSql, theLogging.Abstractionsbump) land there, not here, to keep this slice focused on the store + migrations.🤖 Generated with Claude Code