v1.2.4

Major Features

• Support shard based recovery to improve performance #976

• Improve the stability of Kafka based wal

  • Some enhancements on Kafka client #980 #1005
  • Refactor wal deletion algorithm #1064

• Improve performance of query and write

  • Introduce parquet page filter to accelerate query #664
  • Optimize bloom filter building process to accelerate flush #967 #975

• Support more object store backends

  • Support S3 as object store backend #969
  • Support OceanBase as object store backend(unstable) #887 #970 #971

• Other new features

  • Support hex literal in sql #1030
  • Improve sst-metadata tool for better debugging #1019

What's Changed

• chore: add rationale part in pr template by @jiacai2050 in #957
• chore: remove duplicated metrics by @jiacai2050 in #956
• refactor: replace *join_all with forloop in flush by @baojinri in #947
• fix: capacity should equal to total / part_num by @jiacai2050 in #960
• chore: add log for stream read of remote engine service by @ShiKaiWi in #961
• feat: ignore row group filter when certain column type by @jiacai2050 in #958
• fix: missing metrics when using chain iterator by @ShiKaiWi in #964
• refactor: open shard impl of TableEngine by @Rachelint in #954
• feat: make remote client configurable by @MachaelLee in #948
• feat: use OceanBase as object store by @MachaelLee in #887
• feat: add parquet page filter by @jiacai2050 in #664
• refactor: add grpc write failed counter metric by @baojinri in #963
• feat: expose s3 object store setting by @baojinri in #969
• fix: use uuid as upload_id in obkv object store by @MachaelLee in #970
• refactor: optimize sst filter build to consume less CPU by @zouxiang1993 in #967
• chore: bump xor8 by @jiacai2050 in #972
• feat: use get_batch to implement get_range by @MachaelLee in #971
• refactor: optimize sst iterator and filter build to consume less CPU by @zouxiang1993 in #975
• refactor: disk cache use partition lock by @tanruixiang in #974
• refactor: add generic support to generate hasher by @tanruixiang in #977
• test: add test for hashers by @jiacai2050 in #979
• feat: allow setting multiple kafka boost brokers by @Rachelint in #980
• refactor: add partition num as param in partition lock's init_fn by @tanruixiang in #981
• chore: deny dbg-macro in clippy by @Rachelint in #983
• fix: schema mismatch during write by @chunshao90 in #966
• refactor: function args of object store by @baojinri in #978
• chore: reduce kafka logs by @jiacai2050 in #986
• chore: install git in Dockerfile by @jiacai2050 in #992
• refactor: profiling by @chunshao90 in #989
• feat: support region based wal replay by @Rachelint in #976
• fix: datumkind size by @zouxiang1993 in #994
• chore: add integration test about recovery by @Rachelint in #996
• refactor: add grpc handler metrics by @baojinri in #988
• refactor: avoid grpc forwarding twice by @baojinri in #991
• fix: ensure files can only be picked once by @jiacai2050 in #995
• fix: arrow meta data is lost when decode custom meta data by @ShiKaiWi in #1004
• fix: avoid any updates after table is closed by @ShiKaiWi in #998
• fix: add page index for metadata by @MachaelLee in #1000
• feat: use instead forked rskafka to support limited retry by @Rachelint in #1005
• chore: add logs and metric to recovery by @Rachelint in #1007
• chore: fix logs and style by @Rachelint in #1011
• chore: bump ceresdbproto by @jiacai2050 in #1021
• fix: avoid crash due to empty sql by @tanruixiang in #1024
• feat: add more details about the sst in sst-metadata tool by @zouxiang1993 in #1019
• revert: "fix: add page index for metadata (#1000)" by @ShiKaiWi in #1026
• fix: test_suggest_duration_and_ranges() occasional fail. by @zouxiang1993 in #1028
• chore: bump rust client by @jiacai2050 in #1025
• feat: add page indexes for metadata by @MachaelLee in #1027
• feat: support hex literal by @ShiKaiWi in #1030
• Revert "fix: avoid any updates after table is closed (#998)" by @ShiKaiWi in #1034
• fix: avoid flush too many small sst file by @MachaelLee in #1003
• feat: expose more rocksdb options by @zouxiang1993 in #1033
• feat: add metrics for memtable by @jiacai2050 in #1036
• feat: support opentsdb put api by @zouxiang1993 in #1037
• test: add integration test for opentsdb put api by @zouxiang1993 in #1043
• feat: sst-metadata support sort by @jiacai2050 in #1042
• feat: use dictionary type to store column by @tanruixiang in #993
• fix: compaction support pick by max_seq by @jiacai2050 in #1041
• feat: sql support dictionary column by @tanruixiang in #1049
• chore(deps): bump google.golang.org/grpc from 1.47.0 to 1.53.0 in /integration_tests/sdk/go by @dependabot in #1052
• refactor: add http write metrics by @baojinri in #1045
• feat: memtable bitmap for null by @ShiKaiWi in #1046
• fix: persist sst id by @baojinri in #1009
• fix: fix kafka wal logs deletion and recovery logic by @Rachelint in #1048
• feat: split operations of Cluster and Shard, and serialize operations of Shard by @Rachelint in #1056
• feat: add record batch mem stats by @jiacai2050 in #1058
• fix: fix fully flushed region open in kakfa wal by @Rachelint in #1061
• fix: directly return when found no table datas to replay by @Rachelint in #1062
• feat: warning when query's timestamp is exceeding table's ttl by @tanruixiang in #1054
• refactor: use varint to encode string/bytes length by @baojinri in #1060

Full Changelog: v1.2.2...v1.2.4