Tracking Iteration 2023-11

[ShiKaiWi]

Ease to use

• Improve the success rate for creating new table @ZuLiangWang @ShiKaiWi
  • refactor!: refactor shard version logic #1286
  • refactor!: refactor shard version logic incubator-horaedb-meta#264
• Avoid the primary key duplicate error when use obkv wal implementation @ShiKaiWi OB_ERR_PRIMARY_KEY_DUPLICATE when doing compaction #1205
• Supports idempotence and retry of table creation and deletion. @ZuLiangWang

Performance

Write

• Better load balance according to the actual load of ceresdb-servers @ShiKaiWi
• Reduce the payload size written to the wal by support columnar encoding and compression @ShiKaiWi feat: support write wal logs in columnar format #1179
• Improve the write performance when massive tables exist @ShiKaiWi Reduce the cost of massive stale tables #1281

Query

• Separate the resources used by the queries involving different amount of data @jiacai2050 feat: impl priority runtime for read #1303
• Cache the arrow payload in the memory or on the disk instead of the parquet payload to reduce the frequent decoding cost @jiacai2050 @Rachelint
• Use a read-friendly format for the data in the immutable memtable to reduce the encoding cost during frequent queries @Rachelint

Features

• Support use <database> statement @chunshao90
• Support analyze sql for distributed query @baojinri
• Support query by opentsdb @baojinri feat: implement opentsdb query #1284

Docs

• Update the docs for some new features, e.g. partition table.
• Design docs for the distributed query @Rachelint

[tanruixiang]

If there are any ideas, feel free to bring them up in this issue.

[ShiKaiWi]

The unfinished tasks will be transfered to the Iterration 2023-12.