Skip to content

Latest commit

 

History

History

benches

README.md

solar-bench

Simple benchmarks across different Solidity parser implementations.

Requires cloning submodules: git submodule update --init --checkout.

Run with:

# Criterion
cargo criterion -p solar-bench --bench criterion -- --quiet --format terse |& tee benches/criterion.out
uv --project benches/analyze run benches/analyze/main.py benches/README.md < benches/criterion.out

# iai - requires `valgrind` and `iai-callgrind-runner`
cargo bench -p solar-bench --bench iai

This crate is excluded from the main workspace to avoid compiling it (and its dependencies) when invoking other commands such as cargo test.

Results

You view the Solar-only results on codspeed.io.

The following results were achieved on:

lex Relative Performance

parse Relative Performance

empty (0 LoC, 0 bytes)

Lex

Parser Relative Time LoC/s Bytes/s
solang 16.43x 7.8696 ns N/A N/A
solar 1.00x 115.51 ns N/A N/A

Parse

Parser Relative Time LoC/s Bytes/s
solang 34.59x 103.62 ns N/A N/A
solar 12.50x 285.25 ns N/A N/A
tree-sitter 8.86x 402.46 ns N/A N/A
solc 3.56x 1.0000 µs N/A N/A
slang 1.00x 3.5635 µs N/A N/A

Counter (14 LoC, 258 bytes)

Lex

Parser Relative Time LoC/s Bytes/s
solar 1.55x 903.98 ns 15.50M 285.71M
solang 1.00x 1.3970 µs 10.02M 184.68M

Parse

Parser Relative Time LoC/s Bytes/s
solar 73.18x 2.4130 µs 5.80M 106.92M
solang 22.68x 7.7860 µs 1.80M 33.14M
solc 18.02x 9.8000 µs 1.43M 26.33M
tree-sitter 15.10x 11.697 µs 1.20M 22.06M
slang 1.00x 176.58 µs 79.28K 1.46M

verifier (208 LoC, 11040 bytes)

Lex

Parser Relative Time LoC/s Bytes/s
solar 1.72x 32.986 µs 6.31M 334.69M
solang 1.00x 56.619 µs 3.67M 194.99M

Parse

Parser Relative Time LoC/s Bytes/s
solar 111.35x 84.169 µs 2.47M 131.16M
solang 23.63x 396.61 µs 524.44K 27.84M
solc 15.49x 605.10 µs 343.74K 18.24M
tree-sitter 12.95x 723.89 µs 287.34K 15.25M
slang 1.00x 9.3719 ms 22.19K 1.18M

OptimizorClub (782 LoC, 35905 bytes)

Lex

Parser Relative Time LoC/s Bytes/s
solar 1.64x 106.39 µs 7.35M 337.48M
solang 1.00x 174.52 µs 4.48M 205.74M

Parse

Parser Relative Time LoC/s Bytes/s
solar 100.98x 254.80 µs 3.07M 140.91M
solang 22.05x 1.1667 ms 670.27K 30.77M
solc 16.32x 1.5765 ms 496.04K 22.78M
tree-sitter 12.47x 2.0637 ms 378.93K 17.40M
slang 1.00x 25.729 ms 30.39K 1.40M

UniswapV3 (3189 LoC, 146583 bytes)

Lex

Parser Relative Time LoC/s Bytes/s
solar 1.92x 310.97 µs 10.26M 471.37M
solang 1.00x 595.77 µs 5.35M 246.04M

Parse

Parser Relative Time LoC/s Bytes/s
solar 105.87x 651.14 µs 4.90M 225.12M
solang 22.83x 3.0195 ms 1.06M 48.55M
solc 14.60x 4.7209 ms 675.51K 31.05M
tree-sitter 11.73x 5.8764 ms 542.68K 24.94M
slang 1.00x 68.937 ms 46.26K 2.13M

Solarray (1544 LoC, 35898 bytes)

Lex

Parser Relative Time LoC/s Bytes/s
solar 1.41x 194.99 µs 7.92M 184.10M
solang 1.00x 274.72 µs 5.62M 130.67M

Parse

Parser Relative Time LoC/s Bytes/s
solar 121.81x 526.67 µs 2.93M 68.16M
solc 28.43x 2.2566 ms 684.22K 15.91M
solang 25.90x 2.4768 ms 623.39K 14.49M
tree-sitter 15.79x 4.0618 ms 380.13K 8.84M
slang 1.00x 64.155 ms 24.07K 559.55K

console (1552 LoC, 67315 bytes)

Lex

Parser Relative Time LoC/s Bytes/s
solar 1.53x 275.56 µs 5.63M 244.28M
solang 1.00x 421.40 µs 3.68M 159.74M

Parse

Parser Relative Time LoC/s Bytes/s
solar 109.77x 623.00 µs 2.49M 108.05M
solang 22.02x 3.1052 ms 499.81K 21.68M
solc 19.89x 3.4382 ms 451.40K 19.58M
tree-sitter 13.40x 5.1019 ms 304.20K 13.19M
slang 1.00x 68.386 ms 22.69K 984.34K

Vm (1763 LoC, 91405 bytes)

Lex

Parser Relative Time LoC/s Bytes/s
solar 2.01x 181.55 µs 9.71M 503.47M
solang 1.00x 364.02 µs 4.84M 251.10M

Parse

Parser Relative Time LoC/s Bytes/s
solar 56.35x 321.43 µs 5.48M 284.37M
solang 13.71x 1.3217 ms 1.33M 69.16M
tree-sitter 7.77x 2.3324 ms 755.87K 39.19M
solc 7.36x 2.4618 ms 716.14K 37.13M
slang 1.00x 18.114 ms 97.33K 5.05M

safeconsole (13248 LoC, 397898 bytes)

Lex

Parser Relative Time LoC/s Bytes/s
solar 1.54x 1.4668 ms 9.03M 271.27M
solang 1.00x 2.2593 ms 5.86M 176.12M

Parse

Parser Relative Time LoC/s Bytes/s
solar 82.71x 4.8641 ms 2.72M 81.80M
solang 21.94x 18.336 ms 722.51K 21.70M
tree-sitter 20.05x 20.062 ms 660.35K 19.83M
solc 18.85x 21.347 ms 620.59K 18.64M
slang 1.00x 402.32 ms 32.93K 989.01K

Seaport (19935 LoC, 770547 bytes)

Lex

Parser Relative Time LoC/s Bytes/s
solar 2.07x 1.2604 ms 15.82M 611.35M
solang 1.00x 2.6146 ms 7.62M 294.71M

Parse

Parser Relative Time LoC/s Bytes/s
solar 100.12x 2.3721 ms 8.40M 324.84M
solang 25.80x 9.2062 ms 2.17M 83.70M
solc 11.92x 19.917 ms 1.00M 38.69M
tree-sitter 11.27x 21.073 ms 946.00K 36.57M
slang 1.00x 237.49 ms 83.94K 3.24M