Micro benchmark regularly and display benchmark result

[tiancaiamao]

Feature Request

Is your feature request related to a problem? Please describe:

We do not use the benchmark well.

Benchmarks can guide us the evaluation of the product performance.

• We just cover the typical TPCH TPCC Sysbench test which is too high level.
• We just run the benchmark tests before a new release.

But they are too high level.

Describe the feature you'd like:

I'd like to have some micro benchmarks and run them regularly.

For example, if we have a make bench command to run the micro benchmarks, and the result is collected,
and visualized, then we know the performance regression early.

Describe alternatives you've considered:

For example, there is a scheme benchmark website, https://ecraven.github.io/r7rs-benchmarks/
From which we can see the performance between different scheme compile implementation easily.

Another example is the rust pref https://perf.rust-lang.org/

I'd like TiDB to have something similar, there are may micro benchmarks and the result can be compared by day.

Teachability, Documentation, Adoption, Migration Strategy:

[tiancaiamao]

The performance regression by such kind of changes should be captured by the micro benchmark tikv/client-go#145

[tiancaiamao]

And the point get query performance regression like this one #25573

[tiancaiamao]

I manually run some regression tests and collect the data:

From this data, I can say apparently there are performance regression from 2021-06-08 to 2021-06-11

And from 2021-05-21 to 2021-05-26 BenchmarkBasic result become worse.

[tiancaiamao]

The data is still generated manually, because I want to make sure it works, proof of concept

Later, I'll make it run automatically.

[tiancaiamao]

B.T.W, golang also have a page about perf https://perf.golang.org/, but I dislike it. It's not visualized.

[tiancaiamao]

The data of the last few months is now available here http://www.zenlife.tk:18081/
The final goal is to move it to perf page https://perf.pingcap.com/

[tiancaiamao]

2022-02-14

Performance regression of BenchmarkInsertIntoSelect

and BenchmarkIntegerIndexLookup

[bb7133]

BenchmarkInsertIntoSelect

@tiancaiamao Could you locate the PR/commit that introduces this regression?

[tiancaiamao]

BenchmarkInsertIntoSelect

@tiancaiamao Could you locate the PR/commit that introduces this regression?

#32604 I create an issue for it. @bb7133