Convert JSON files to Apache Arrow. You may also be interested in csv2arrow, json2parquet, or csv2parquet.
You can get the latest releases from https://github.com/domoritz/json2arrow/releases/.
cargo install json2arrow
USAGE:
json2arrow [FLAGS] [OPTIONS] <JSON> [ARROW]
ARGS:
<JSON> Input JSON file
<ARROW> Output file, stdout if not present
FLAGS:
-h, --help Prints help information
-v, --verbose Print the schema to stderr
-V, --version Prints version information
OPTIONS:
-m, --max-read-records <max-read-records>
The number of records to infer the schema from. All rows if not present
Since we use teh Arrow JSON loader, we are limited to what it supports. Right now, it supports JSON line-delimited files.
{ "a": 42, "b": true }
{ "a": 12, "b": false }
{ "a": 7, "b": true }
To format the code, run
cargo clippy && cargo fmt