cb-stream

This is a Rust application that uses WebSockets to stream real-time data from Coinbase via their Market Data feeds. I am currently in the process of learning Rust and using this project as another step in my learning experience. I am not fully sure what the end result of this project will look like but I would like to take the event data from Coinbase, transform + normalize it, and then visualize/present in some way to the end user. My initial thought is to create some kind of interactive program where someone can replay events, view historical behavior of crypto prices, etc.

Coinbase Market Data Docs: https://docs.cdp.coinbase.com/exchange/websocket-feed/overview

Usage

cargo build
cargo run                   # Run the data stream service with default values from env
cargo run -- <CLI args>     # Run the data stream based on passed in args
cargo run -- <-h/--help>    # Pull up the help dialog for the CLI

Current Features

• Subscribe to any valid channel available via Coinbase Market Data feeds
  • e.g. ticker, heartbeat, matches, etc.
• Subscribe to any valid product available via Coinbase Market Data feeds
  • e.g. BTC-USC, ETH-USD, SOL-USD
• Channel + product subscriptions can be defined as env vars or passed in as CLI args
• Events are streamed via WebSockets and written to stdout and a logs folder as JSON files
  • Events get logged as individual file
  • You can use the included combine-event.sh shell script to quickly consolidate logs into one text document
• Provided shell script (rtds-pg.sh) allows you to quickly spin up a Postgres container to use with the service

Future Features

• Integration with Postgres to persist event data for use in replay, data visualization, etc.
• Backfill job to fill in any gaps in the event history due to any issues during application execution

Postgres Quickstart Script

This repo includes rtds-pg.sh to spin up + manage a Postgres container for development. You will need to create an .env file to define various parameters. Refer to .env.example for a reference on what values need to be filled in.

Common commands:

./rtds-pg.sh build    # pull image + create container
./rtds-pg.sh start    # start existing container
./rtds-pg.sh stop     # stop container
./rtds-pg.sh status   # show container + health
./rtds-pg.sh logs     # tail container logs
./rtds-pg.sh psql     # open psql inside container
./rtds-pg.sh url      # print connection URL
./rtds-pg.sh del      # remove container only
./rtds-pg.sh fdel     # remove container + data volume