Skip to content

Latest commit

 

History

History
29 lines (23 loc) · 713 Bytes

README.md

File metadata and controls

29 lines (23 loc) · 713 Bytes
Raw

Yellowbrick Test

This project initalizes Kafka, writes 3million messages into a Kafka topic via the datagen connector and then sinks those messages into a postgres JDBC sink

Steps

create an .env file with the following variables for connecting to the postgres database:

PGHOST=
PGPORT=
PGDATABASE=
PGUSER=
PGPASSWORD=
  1. Start the docker container (starts zookeper, kafka brokers, schema registry, kafka-connect, C3, KSQLDB, postgres)
docker-compose up -d
  1. Create the initial topics & start the datagen connector
scripts/create-kafka-records.sh
  1. Start the JDBC sink for postgres, it will track the records and report records per second
scripts/sink-and-measure.sh