Move some existing config options to key-value based configuration #2756
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
PR #2754 adds an extensible configuration mechanism based on key-value pairs.
This issue proposes to migrate the following existing configuration options in SessionConfig to this approach:
/// Default batch size while creating new batches, it's especially useful
/// for buffer-in-memory batches since creating tiny batches would results
/// in too much metadata memory consumption.
pub batch_size: usize,
/// Number of partitions for query execution. Increasing partitions can increase concurrency.
pub target_partitions: usize,
/// Default catalog name for table resolution
default_catalog: String,
/// Default schema name for table resolution
default_schema: String,
/// Whether the default catalog and schema should be created automatically
create_default_catalog_and_schema: bool,
/// Should DataFusion provide access to `information_schema`
/// virtual tables for displaying schema information
information_schema: bool,
/// Should DataFusion repartition data using the join keys to execute joins in parallel
/// using the provided `target_partitions` level
pub repartition_joins: bool,
/// Should DataFusion repartition data using the aggregate keys to execute aggregates in parallel
/// using the provided `target_partitions` level
pub repartition_aggregations: bool,
/// Should DataFusion repartition data using the partition keys to execute window functions in
/// parallel using the provided `target_partitions` level
pub repartition_windows: bool,
/// Should DataFusion parquet reader using the predicate to prune data
pub parquet_pruning: bool,Describe the solution you'd like
As described
Describe alternatives you've considered
None
Additional context
None