v6.5 is 6x slower than v6.1.3 on analyze for tpch 50g

[dbsid]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

set tidb_build_stats_concurrency=32; set tidb_distsql_scan_concurrency=30; set tidb_index_serial_scan_concurrency=32; analyze table lineitem; analyze table orders; analyze table part; analyze table partsupp; analyze table supplier; analyze table customer; analyze table region; analyze table nation;

2. What did you expect to see? (Required)

no regression

3. What did you see instead (Required)

4. What is your TiDB version? (Required)

v6.5

[xuyifangreeneyes]

After binary searching git log, we found that #35975 seems to be the root cause. It sets concurrency to 2 for the request whose KeepOrder is true. Analyze request needs to set KeepOrder to true since it needs to calculate the correlations between the primary key and other columns.

[xuyifangreeneyes]

/cc @tiancaiamao

[xuyifangreeneyes]

It seems that analyze request doesn't need to keep order. Correlation is calculated by samples. Even if we don't keep order when sending requests, we can sort samples before calculating correlation. cc @winoros