Add workflow with criterion execution benches

[FrancoGiachetta]

Add workflow with criterion execution benches

Closes #NA

Our benches currently use hyperfine, which takes compilation time into account. This PR adds criterion benches, which measures execution time.

Introduces Breaking Changes?

No.

• Created PR in sequencer
• Created PR in starknet-replay
• Updated the starknet-blocks.yml workflow to use these PRs.

These PRs should be merged after this one right away, in that order.

Checklist

• Linked to Github Issue.
• Unit tests added.
• Integration tests added.
• This change requires new documentation.
  • Documentation has been added/updated.

[github-actions]

Benchmark results Main vs HEAD.

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base dict_insert.cairo (JIT)	2.596 ± 0.027	2.559	2.636	1.03 ± 0.02
base dict_insert.cairo (AOT)	2.515 ± 0.034	2.459	2.566	1.00

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head dict_insert.cairo (JIT)	2.855 ± 0.079	2.750	2.986	1.08 ± 0.04
head dict_insert.cairo (AOT)	2.634 ± 0.060	2.519	2.693	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base dict_snapshot.cairo (JIT)	2.345 ± 0.026	2.306	2.389	1.05 ± 0.02
base dict_snapshot.cairo (AOT)	2.231 ± 0.045	2.179	2.303	1.00

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head dict_snapshot.cairo (JIT)	2.429 ± 0.028	2.376	2.464	1.01 ± 0.03
head dict_snapshot.cairo (AOT)	2.417 ± 0.059	2.331	2.523	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base factorial_2M.cairo (JIT)	2.678 ± 0.032	2.653	2.759	1.00 ± 0.02
base factorial_2M.cairo (AOT)	2.672 ± 0.039	2.634	2.759	1.00

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head factorial_2M.cairo (JIT)	2.939 ± 0.095	2.822	3.084	1.05 ± 0.04
head factorial_2M.cairo (AOT)	2.787 ± 0.036	2.731	2.850	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base fib_2M.cairo (JIT)	2.207 ± 0.044	2.146	2.281	1.01 ± 0.03
base fib_2M.cairo (AOT)	2.183 ± 0.039	2.143	2.241	1.00

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head fib_2M.cairo (JIT)	2.415 ± 0.077	2.336	2.594	1.04 ± 0.05
head fib_2M.cairo (AOT)	2.311 ± 0.073	2.202	2.426	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base linear_search.cairo (JIT)	2.444 ± 0.035	2.382	2.501	1.07 ± 0.03
base linear_search.cairo (AOT)	2.284 ± 0.042	2.204	2.325	1.00

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head linear_search.cairo (JIT)	2.494 ± 0.062	2.412	2.616	1.04 ± 0.04
head linear_search.cairo (AOT)	2.393 ± 0.054	2.327	2.489	1.00

Base

Command	Mean [s]	Min [s]	Max [s]	Relative
base logistic_map.cairo (JIT)	2.525 ± 0.029	2.481	2.564	1.06 ± 0.03
base logistic_map.cairo (AOT)	2.373 ± 0.054	2.277	2.440	1.00

Head

Command	Mean [s]	Min [s]	Max [s]	Relative
head logistic_map.cairo (JIT)	2.664 ± 0.085	2.556	2.868	1.02 ± 0.04
head logistic_map.cairo (AOT)	2.607 ± 0.078	2.478	2.710	1.00

[github-actions]

Benchmarking results

Benchmark for program dict_insert

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	10.878 ± 0.058	10.837	11.032	4.40 ± 0.05
cairo-native (embedded AOT)	2.475 ± 0.026	2.443	2.528	1.00
cairo-native (embedded JIT using LLVM's ORC Engine)	2.551 ± 0.015	2.526	2.580	1.03 ± 0.01

Benchmark for program dict_snapshot

Open benchmarks

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
Cairo-vm (Rust, Cairo 1)	530.4 ± 10.9	516.8	550.8	1.00
cairo-native (embedded AOT)	2168.1 ± 48.0	2121.8	2288.5	4.09 ± 0.12
cairo-native (embedded JIT using LLVM's ORC Engine)	2257.7 ± 18.1	2230.1	2286.2	4.26 ± 0.09

Benchmark for program factorial_2M

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	4.734 ± 0.012	4.715	4.757	1.82 ± 0.03
cairo-native (embedded AOT)	2.605 ± 0.042	2.562	2.676	1.00
cairo-native (embedded JIT using LLVM's ORC Engine)	2.629 ± 0.028	2.590	2.688	1.01 ± 0.02

Benchmark for program fib_2M

Open benchmarks

Command	Mean [s]	Min [s]	Max [s]	Relative
Cairo-vm (Rust, Cairo 1)	4.685 ± 0.013	4.666	4.699	2.18 ± 0.03
cairo-native (embedded AOT)	2.146 ± 0.031	2.111	2.205	1.00
cairo-native (embedded JIT using LLVM's ORC Engine)	2.180 ± 0.019	2.156	2.212	1.02 ± 0.02

Benchmark for program linear_search

Open benchmarks

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
Cairo-vm (Rust, Cairo 1)	575.5 ± 6.8	568.7	589.2	1.00
cairo-native (embedded AOT)	2218.3 ± 19.9	2197.2	2251.7	3.85 ± 0.06
cairo-native (embedded JIT using LLVM's ORC Engine)	2344.0 ± 20.4	2312.9	2376.8	4.07 ± 0.06

Benchmark for program logistic_map

Open benchmarks

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
Cairo-vm (Rust, Cairo 1)	383.5 ± 3.5	377.9	391.1	1.00
cairo-native (embedded AOT)	2288.9 ± 17.3	2257.9	2323.8	5.97 ± 0.07
cairo-native (embedded JIT using LLVM's ORC Engine)	2466.4 ± 28.7	2435.4	2520.7	6.43 ± 0.10

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.30%. Comparing base (6f1954d) to head (55aaaae).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1416   +/-   ##
=======================================
  Coverage   81.30%   81.30%           
=======================================
  Files         105      105           
  Lines       25741    25741           
=======================================
  Hits        20929    20929           
  Misses       4812     4812           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.