Feature Request
next generation TiDB will store data on cloud storage, such as S3/GCS, and will use cloud infrastructure to make sure high availability, and TiKV will load data as needed.
in this case, for add-index/import-into on global sort, we need to adapt this change, the major change is:
- the KV encoding, global-sorting, pause region scheduler, region split and scatter, part is the same as current code path
- when write KV pair, we write to a servitization component, let's call it
tikv-worker, which will accept the KV pair and write a SST file to cloud storage, and return the filename back
- note: we only need write one SST file, not 3
- add-index/import-into will call ingest API on TiKV using the filename as part of the component
currently, we will implememt this into a feature branch, and use a separate binary for NG TiDB.
Subtasks
nextgen build
ingesting in nextgen
senity check for nextgen
test related
Feature Request
next generation TiDB will store data on cloud storage, such as S3/GCS, and will use cloud infrastructure to make sure high availability, and TiKV will load data as needed.
in this case, for add-index/import-into on global sort, we need to adapt this change, the major change is:
tikv-worker, which will accept the KV pair and write a SST file to cloud storage, and return the filename backcurrently, we will implememt this into a feature branch, and use a separate binary for NG TiDB.
Subtasks
nextgen build
ingesting in nextgen
senity check for nextgen
test related