Sequin is a tool for capturing changes and streaming data out of your Postgres database.
Sequin is great for:
- Replicating data from your existing tables to other apps, databases, caches, materialized views, or frontend clients.
- Building event driven workflows such as triggering side effects when data in Postgres changes.
Sequin itself is built on Postgres. It uses a logical replication slot to detect changes and internal tables to store consumer state. Without Sequin, you'd need to cobble together tools like Debezium and Kafka.
Sequin is a standalone Docker container that you can deploy in front of your Postgres database. Or, you can use our hosted offering.
Sequin is open source/MIT. To help us make this project great, tell us what you're building in our Discord Server.
- Never miss a record or change: Sequin ensures all database changes are delivered and processed exactly once by consumers.
- SQL-based routing: Filter and route messages to consumers using SQL
whereconditions. - Replays: Rewind consumers to any row on your table. Or republish select messages that match a SQL query.
- Start anywhere: Consumers can start processing records from any point in a table.
- Bring your database: Sequin is not an extension. It works with any Postgres database version 12+.
- No PL/pgSQL: Define business logic in the language of your choice and in your application.
- Transforms (coming soon!): Transform message payloads by writing functions in Lua, JavaScript, or Go.
The quickest way to get started is to create an account on Sequin Cloud. Follow the instructions in the app to start streaming your data in a couple minutes.
Our cloud quickstart and database setup guides are helpful resources.
If you prefer to self-host, follow our self-hosted quickstart.
You can run Sequin in its own Docker container or as a sidecar container in your existing deployment.
Replication: Sync data to other systems to update search indexes, invalidate caches, and generate denormalized views.
Trigger side-effects: Never miss a change in your database for processing by other systems.
Fan out: Broadcast events, distribute workloads, and decouple services.
Sequin keeps your data in your Postgres database. You can use your existing database in a new way without copying the data to a new system or mastering a new technology.
Sequin connects to any Postgres database. To stream data, you'll create Sequences for each table you want to stream. Sequences present a strictly ordered view of rows from one or more tables. Then, you can use this Sequence to consume rows by:
- Using the Consume API
- Receiving webhooks
- Using the Sync API (coming soon)
As rows are inserted or updated, Sequin will redeliver them to consumers until acknowledged.
With WAL Pipelines, you can capture discrete changes to your tables, including OLD values for updates and hard-deletes. Sequin will write changes to an event log table in your database, so you can stream these changes with Sequin.
Sequin comes with a web console/UI for configuration:
- Connect any Postgres database to Sequin (Sequin uses logical replication).
- Add Sequences and consumers to tables you want to stream.
- Consume messages using Sequin's APIs.
You can configure Sequin as code using TOML config files (coming soon).
Sequin efficiently captures changes using logical replication. This adds virtually no overhead to the performance of your database. If your database can handle the transaction, so can Sequin with minimal latency.
Postgres Performance is highly dependent on machine resources. But to give you an idea, a db.m5.xlarge RDS instance (4 vCPU, 16 GB RAM, $260/mo) can handle inserts at 5,000 messages/second, with bursts up to 10k messages/second.
Sequin vs Kafka
Apache Kafka is a distributed event streaming platform. Kafka is designed for very high throughput and horizontal scalability.
You can use Sequin to turn a Postgres table into something that behaves like a Kafka topic. You can create new consumers that process messages in order, starting at any offset in the table you specify. Because all your data lives at rest in Postgres, the data model is easy to understand and work with.
Sequin's consumer pattern is simpler than Kafka's. Kafka uses partitions and offsets for concurrency, whereas Sequin uses a message queue pattern similar to SQS. This means concurrency is flexible and you can scale workers up and down without making any configuration changes.
While Kafka may be necessary for very high throughput use cases (logs or metrics, millions of messages per second), Postgres and Sequin are able to handle a lot of use cases with a lot less complexity (even modest Postgres instances can handle tens of thousands of messages per second).
Sequin vs Debezium + Kafka
Debezium is a change data capture (CDC) tool for databases like Postgres. It requires an external messaging system like Kafka to operate.
Like Sequin, you can use Debezium + Kafka to replicate data or build event processing workflows.
Debezium is a complex system that requires a lot of setup and configuration.
Sequin is simpler to setup and operate, yet is quickly becoming as comprehensive as Debezium. Sequin comes with a much more comprehensive UI for configuration and monitoring. And Sequin doesn't require another messaging system like Kafka to learn and operate. With WAL Pipelines you can capture the same changes that Debezium does.
Sequin vs PG Triggers
PG Triggers are database functions that execute in response to certain database events. Postgres triggers must be written in PL/pgSQL, a special procedural language for Postgres. And they are limited to simple operations like inserting, updating, and deleting rows.
Sequin sends changes over HTTP. Like Postgres triggers, Sequin guarantees exactly-once processing. But the HTTP interface means you can you can handle changes in your application code. Just like other advanced queuing systems, Sequin lets you fan out changes to multiple services or applications. And you can write business logic in the programming language of your choice (no PL/pgSQL!)
Sequin vs LISTEN / NOTIFY
Postgres' LISTEN/NOTIFY is a pub/sub system. When changes happen in your database, you can send notifications to a channel in Postgres. Other processes can listen for notifications on that channel and react to changes.
LISTEN/NOTIFY offers at-most-once delivery. If a consumer is not subscribed when a notification is sent, or a consumer fails to process a notification, the notification is lost. So LISTEN/NOTIFY is limited to simple use cases.
Sequin sends changes over HTTP with exactly-once processing. This gives you a much more robust system to build on top of.
Sequin vs Amazon SQS
Amazon Simple Queue Service (SQS) is a message queuing service. It offers exactly-once processing over an HTTP interface.
Sequin's HTTP pull interface is a lot like SQS's HTTP pull interface. Except, Sequin isn't really a queue; it's better thought of as a stream. Because Sequin is streaming your Postgres tables, messages aren't deleted after they're processed. This means you get more advanced capability, like rewind and replay.
Sequin is open-sourced with an MIT license. The project is just getting started, so the best way to contribute right now is to open an issue or join the Discord Server and let us know what you want to build.
![@dependabot[bot]](https://avatars.githubusercontent.com/in/29110?s=64&v=4){kind=small}
