Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

coprocessor: do not ask unnecessary columns' info to avoid read values in TiKV if necessary #56004

Open
AndreMouche opened this issue Sep 11, 2024 · 1 comment
Labels
component/executor report/customer Customers have encountered this bug. type/enhancement The issue or PR belongs to an enhancement.

Comments

@AndreMouche
Copy link
Contributor

Enhancement

drop table if exists t1, t2;

create table t1 (a int, b varchar(10), pad1 varchar(2048), pad2 varchar(2048), pad3 varchar(2048), primary key (a));
create table t2 (a int, b varchar(10), pad1 varchar(2048), pad2 varchar(2048), pad3 varchar(2048), primary key (b, a));

insert into t1 values (1, 1, repeat('a', 2048), repeat('b', 2048), repeat('c', 2048));
insert into t1 select a + 1, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 2, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 4, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 8, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 16, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 32, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 64, b, pad1, pad2, pad3 from t1;
insert into t1 select a + 128, b, pad1, pad2, pad3 from t1;
insert into t2 select * from t1;

analyze table t1;
analyze table t2;

explain analyze select count(1) from t1;
explain analyze select count(1) from t2;

The two select count(*) statement should have the similar process key size.

Select from t1 ...total_process_keys: 256, total_process_keys_size: 6912...
Select from t2 ...total_process_keys: 256, total_process_keys_size: 1586944...

The t2 statement reads row value as well, so the total_process_keys_size is over hundred time of t1's.

Tikv determines whether to scan the value based on the columns_info in from the request. it will scan the value if it need any column from no-primary-key,related check is here:
https://github.com/tikv/tikv/blob/a4c0ea1657b3d939da51ea1cbbe77aff94bb60d3/components/tidb_query_executors/src/table_scan_executor.rs#L78-L83

Meanwhile, the column_info here comes from the request, the related proto are here:

https://github.com/pingcap/tipb/blob/e46e4632bd2b8c28a1a5f0986513bec8e25984e9/proto/executor.proto#L140-L154

message TableScan {
	optional int64 table_id = 1 [(gogoproto.nullable) = false];
	repeated ColumnInfo columns = 2;
	optional bool desc = 3 [(gogoproto.nullable) = false];
	repeated int64 primary_column_ids = 4;
	optional EngineType next_read_engine = 5 [(gogoproto.nullable) = false]; // which engine we should in next step, only used by tiflash
	repeated KeyRange ranges = 6 [(gogoproto.nullable) = false]; // For global read in join, we must point out the key ranges when we don't have the region info.
	repeated int64 primary_prefix_column_ids = 7;
	optional bool keep_order = 8;
	optional bool is_fast_scan = 9; // fast_scan is a feature only provided by TiFlash (but not TiKV).
	repeated Expr pushed_down_filter_conditions = 10; // conditions that are pushed down to storage layer, only used by TiFlash.
	optional ANNQueryInfo ann_query = 13; // only used by TiFlash
	repeated RuntimeFilter runtime_filter_list = 11; // only used by TiFlash
	optional int32 max_wait_time_ms = 12  [(gogoproto.nullable) = false];
}
@AndreMouche AndreMouche added type/enhancement The issue or PR belongs to an enhancement. component/executor labels Sep 11, 2024
@AndreMouche AndreMouche changed the title coprocessor: do not ask columns' info to avoid read values in TiKV if necessary coprocessor: do not ask unnecessary columns' info to avoid read values in TiKV if necessary Sep 11, 2024
@seiya-annie
Copy link

/report customer

@ti-chi-bot ti-chi-bot bot added the report/customer Customers have encountered this bug. label Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/executor report/customer Customers have encountered this bug. type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

No branches or pull requests

2 participants