Decrease SST metadata size

[jiacai2050]

Describe This Problem

We currently use parquet as our underlying file format, and we find the metadata of one SST is very large. Eg:

size:348.675M, metadata:40.142M, kv:38.175M, filter:28.525M, row_num:7038949

Ideally SST's metadata should be cached to improve query perf, so we need to find solutions to decrease metadata size to avoid OOM in production.

Proposal

• Reduce xor filter size
  • In practical double/bool column and key column could skip xor filter, since mix/max filter works well for them. feat: ignore row group filter when certain column type #958
  • Choose another filter when distinct values are less than 10.
• Reduce unnecessary base64 encoding when write metadata, it can cause size increase more than 25%. This task depends on pass bytes directly to parquet's KeyValue arrow-rs#4317

Additional Context

Xor filter size for different key num

key_num:0, len:54, byte_per_key:54
key_num:1, len:57, byte_per_key:57
key_num:5, len:63, byte_per_key:12.6
key_num:10, len:69, byte_per_key:6.9
key_num:100, len:177, byte_per_key:1.77
key_num:1000, len:1284, byte_per_key:1.284
key_num:2000, len:2514, byte_per_key:1.257
key_num:3000, len:3744, byte_per_key:1.248
key_num:4000, len:4974, byte_per_key:1.2435
key_num:8000, len:9894, byte_per_key:1.23675

This snippet show when key are less than 10, the cost of per key is relatively high, maybe we could use hash(u16) to store them.