Skip to content
/ dozer Public

Dozer is a real-time data movement tool that leverages CDC from various sources and moves data into various sinks.

License

Notifications You must be signed in to change notification settings

getdozer/dozer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Data APIs, done right!

⚡️ Open-source platform to build, publish and manage blazing-fast real-time data APIs in minutes ⚡️

stars CI Coverage Status Doc reference Join on Discord License


Quick Start

Download the latest stable binary or source code for a quick start. Check out our latest release details here.

Using binary

curl -L "https://drive.google.com/uc?export=download&id=1-mhZUV4HK2agyPwqpOzUWntqQzhjEJPN&confirm=9iBg" | bash
cd dozer
./dozer

Using docker

curl -L "https://drive.google.com/uc?export=download&id=1-mhZUV4HK2agyPwqpOzUWntqQzhjEJPN&confirm=9iBg" | bash -s -- -d
cd dozer
docker compose -f <path-to-yaml-config> up

Why use Dozer?

  • Create blazing fast end to end APIs in minutes with a simple configuration.
  • Build and rapidly iterate on customer facing data apps.
  • Extend Dozer with custom connectors, operators and Api transformations using WASM.
  • Built in Rust with performance and extensibility in mind.

End-to-end Examples

What can you build with Dozer?

  1. Hypercharge your Postgres by offloading read APIs to Dozer
  2. Real time ML predictions deployed as APIs from Snowflake
  3. Real time Ethereum Stats published as a grafana dashboard

Check out this module for above end-to-end examples.


Features

  • Connect your sources
    • Import real time data from Postgres as CDC, Snowflake Table Stream etc.
    • Create your own connector using Rust
    • Automatic schema evolution and validation
  • Transform in REAL-TIME
    • Use SQL to perform joins, aggregations and filter operations in real time across sources.
    • Use it like an ORM; Map relational data to object entities using Dozer SQL extensions
    • Build custom functions for aggregation, selection etc. using WASM
  • Optimize for serving
    • Define indices with a simple configuration
    • Support for multiple indices such as Inverted, Full Text, Compound, Geo (Coming soon!) etc.
    • Apply filter and sort operations on cached data
    • Support for Push and Pull queries
  • Publish blazing fast APIs
    • gRPC and REST APIs automatically generated
    • Protobuf an Open API documentation
    • TypeSafe APIs
    • Realtime Streaming

Contributing

Contributors / Developers

Build Dependencies

  • Rust
  • protoc latest release on your PATH
  • sqlite3 (sudo apt install libsqlite3-dev on Ubuntu)
  • openssl (brew install pkg-config openssl on MacOS)

Please refeer to this module on how to test/build/implement a new connector.