Move Benchmarks From Tools/ to the pyperformance Repo

[ericsnowcurrently]

(See https://discuss.python.org/t/remove-outdated-tools-scripts-scripts/19571/10.)

The Tools directory currently has some benchmarks in it:

• ccbench
• iobench
• importbench
• stringbench
• maybe others?

It would probably make sense to move them to the pyperformance repo.

CC @rhettinger @gvanrossum

[arhadthedev]

maybe others?

Tool/scripts/var_access_benchmark.py

[rhettinger]

Why put them somewhere else? These are used during CPython development to quickly test the effect of changes. If you put them in another repo, they just get harder to access — I would likely never use them again or would have to keep a local copy.

IIRC the original purpose of the Tools directly was as a convenient place for us to keep scripts that we use during development.

[gvanrossum]

I created the “faster Python” project and we definitely don’t use these to assess performance improvements.

[ericsnowcurrently]

Regardless of the value of keeping these in the Tools directory, adding these micro benchmarks to pyperformance is still likely worth doing.

(FWIW, I have no real interest either way in the ongoing Tools/ cleanup.)

[corona10]

@ericsnowcurrently

I concur with Eric's comment.
As the #98152 author (even if it was duplicated).
The Faster CPython team measures performance regression for every iteration so to make their task more effective we should centralize benchmark suites as possible.
And also nowadays most of performance people are looking pyperformance benchmarks instead of running the manual bench suites under Tools directory.
If we port them into pyperformance benchmarks we can get the diff result for every iteration from https://github.com/faster-cpython/ideas/tree/main/benchmark-results.

Yeah, accessibility from some people can be degraded but I think that overall gain will be positive if we move them into pyperformance projects and I believe that we should reduce fragmentation as possible.

[gvanrossum]

If someone submits a PR to pyperformance can they please link to it here?

[vstinner]

I proposed #98214 to just remove Tools/scripts/startuptime.py: there are other existing tools to measure Python startup time.