Skip to content

Postgres change data capture to streams and queues like Kafka, SQS, HTTP endpoints, and more

License

Notifications You must be signed in to change notification settings

sequinstream/sequin

Repository files navigation

Sequin

Send Postgres changes anywhere

Docs License: MIT

Sequin Cloud · Documentation · Website

What is Sequin?

Sequin sends Postgres records and changes to your applications and services. It’s designed to never miss an insert, update, or delete and provide exactly-once processing of all changes.

You can receive changes via HTTP push (webhooks) or pull (think SQS).

Killer features

  • Never miss a message: Sequin delivers messages from your database to consumers until they are acknowledged (i.e. exactly-once processing guarantees).
  • SQL-based routing: Filter and route messages to consumers using SQL where conditions.
  • Replays: Rewind consumers to any row on your table. Or republish select messages that match a SQL query.
  • Backfills: Consumers can queue up records from this moment forward or for the entire table.
  • Simple HTTP API: Messages are delivered via a simple HTTP push or pull interface for easy integration.
  • Bring your database: Sequin is not an extension. It works with any Postgres database version 12+.
  • No plpgsql: Define business logic in the language of your choice and in your application.
  • Transforms (coming soon!): Transform message payloads by writing functions in Lua, JavaScript, or Go.

Cloud

Try Sequin Cloud now →

How Sequin works

Sequin keeps your data in your Postgres database. You can use your existing database in a new way without copying the data to a new system or mastering a new technology.

Sequin connects to any Postgres database via the Write Ahead Log (WAL). It captures changes to your Postgres tables, turning them into a stream of messages. Sequin buffers messages to ensure they are delivered and allow for replays and rewinds. You can read those messages over an HTTP interface similar to SQS, or Sequin can push changes to you via webhooks.

Sequin is open source. You can run Sequin in its own Docker container or as a sidecar container in your existing deployment.

Sequin comes with a web console/UI for configuration:

  1. Connect any Postgres database to Sequin (Sequin uses logical replication).
  2. Select the tables, actions, and SQL filters that publish messages to your consumers.
  3. Consume messages using HTTP pull and push interfaces. (You can use our SDKs.)

In production, you can configure Sequin as code using Terraform.

Benchmarks

Sequin efficiently captures changes using the Write Ahead Log. This adds virtually no overhead to the performance of your database. If your database can handle the transaction, so can Sequin with minimal latency.

Postgres Performance is highly dependent on machine resources. But to give you an idea, a db.m5.xlarge RDS instance (4 vCPU, 16 GB RAM, $260/mo) can handle inserts at 5,000 messages/second, with bursts up to 10k messages/second.

Use cases

Trigger side-effects: Never miss a change in your database for processing by other systems. Queue email

Fan out: Add queue capabilities to your database to broadcast events, distribute workloads, and decouple services. Fan out order

Replication: Sync data to other systems to update search indexes, invalidate caches, and generate denormalized views. Sync

Compare Sequin

Feature Sequin PG triggers LISTEN / NOTIFY Supabase Webhooks Amazon SQS
Trigger guarantees Transactional Transactional Transactional Transactional N/A
Delivery guarantees Exactly-once Exactly-once At-most-once At-least-once Exactly-once
Order guarantees FIFO, strict by PK FIFO ❌ No ❌ No FIFO option
Replay ✅ Yes ❌ No ❌ No ❌ No ❌ No
Filtering ✅ Yes ✅ Yes ✅ Yes ✅ PG triggers N/A
Transformation ✅ LUA ❌ No ❌ No ❌ No ❌ No
Backfills ✅ Yes ❌ No ❌ No ❌ No N/A
Interface HTTP pull
HTTP push
plpgsql Postgres client HTTP push HTTP pull
Observability Sequin console PG logging* PG logging* Supabase dashboard AWS console
Performance WAL, minimal overhead Serial ops / row^ Minimal overhead Serial ops / row^ N/A

* PG logging: You can configure logging in your database, but nothing is built in. Generally hard to see the state of any given side-effect.

^ Serial ops / row: Postgres triggers (which power Supabase Webhooks) run serially within each transaction (and block) and can add significant overhead.

Documentation

For full documentation, visit sequinstream.com/docs.

Quick start

Sequin is available as a cloud version or self-hosted. See the following docs to get up and running quickly:

SDKs

Use Sequin's SDKs to dynamically create new consumers and work with messages:

Contribute

Sequin is open-sourced with an MIT license. The project is just getting started, so the best way to contribute right now is to open an issue or send us a note: support@sequinstream.com