|
1 | 1 | # Datagen CLI
|
2 | 2 |
|
3 |
| -This command line interface application allows you to take schemas defined in JSON (`.json`), Avro (`.avsc`), or SQL (`.sql`) and produce believable fake data to Kafka in JSON or Avro format or to Postgres. |
| 3 | +This command line interface application allows you to take schemas defined in JSON (`.json`), Avro (`.avsc`), or SQL (`.sql`) and produce believable fake data to Kafka in JSON or Protobuf format or Avro format or to Postgres. |
4 | 4 |
|
5 | 5 | The benefits of using this datagen tool are:
|
6 | 6 | - You can specify what values are generated using the expansive [FakerJS API](https://fakerjs.dev/api/) to craft data that more faithfully imitates your use case. This allows you to more easily apply business logic downstream.
|
@@ -85,7 +85,7 @@ Fake Data Generator
|
85 | 85 | Options:
|
86 | 86 | -V, --version output the version number
|
87 | 87 | -s, --schema <char> Schema file to use
|
88 |
| - -f, --format <char> The format of the produced data (choices: "json", "avro", "postgres", "webhook", default: "json") |
| 88 | + -f, --format <char> The format of the produced data (choices: "json", "avro", "postgres", "webhook", "proto", default: "json") |
89 | 89 | -n, --number <char> Number of records to generate. For infinite records, use -1 (default: "10")
|
90 | 90 | -c, --clean Clean (delete) Kafka topics and schema subjects previously created
|
91 | 91 | -dr, --dry-run Dry run (no data will be produced to Kafka)
|
@@ -213,7 +213,11 @@ Here is the general syntax for a JSON input schema:
|
213 | 213 | {
|
214 | 214 | "_meta": {
|
215 | 215 | "topic": "<my kafka topic>",
|
216 |
| - "key": "<field to be used for kafka record key>" , |
| 216 | + "key": "<field to be used for kafka record key>" , |
| 217 | + "proto": { |
| 218 | + "dir": "<directory with protobuf schemas>", |
| 219 | + "schema": "<protobfuf message schema name>" |
| 220 | + }, |
217 | 221 | "relationships": [
|
218 | 222 | {
|
219 | 223 | "topic": "<topic for dependent dataset>",
|
|
0 commit comments