Skip to content

kostya/benchmarks

Repository files navigation

Table of Content

Overview

The benchmarks follow the criteria:

  • They are written as the average software developer would write them, i.e.

    • The algorithms are implemented as cited in public sources;
    • The libraries are used as described in the tutorials, documentation and examples;
    • The used data structures are idiomatic.
  • The used algorithms are similar between the languages (as the reference implementations), variants are acceptable if the reference implementation exists.

  • All final binaries are releases (optimized for performance if possible) as debug performance may vary too much depending on the compiler.

My other benchmarks: jit-benchmarks, crystal-benchmarks-game

Measurements

The measured values are:

  • time spent for the benchmark execution (loading required data and code self-testing are not measured);
  • memory consumption of the benchmark process, reported as base + increase, where base is the RSS before the benchmark and increase is the peak increase of the RSS during the benchmark;
  • energy consumption of the CPU package during the benchmark: PP0 (cores) + PP1 (uncores like GPU) + DRAM. Currently, only Intel CPU are supported via the powercap interface.

All values are presented as: median±median absolute deviation.

UPDATE: 2025-08-17

Test Cases

Brainfuck

Testing brainfuck implementations using two code samples (bench.b and mandel.b). Supports two mode:

  • Verbose (default). Prints the output immediately.
  • Quiet (if QUIET environment variable is set). Accumulates the output using Fletcher-16 checksum, and prints it out after the benchmark.

Brainfuck

bench.b

Language Time, s Memory, MiB Energy, J
Scala (Staged) 0.409±0.011 229.62±03.27 + 7.00±01.44 26.31±00.71
Racket (Staged) 0.885±0.000 101.37±01.65 + 0.06±00.06 34.39±00.17
V/gcc 0.985±0.001 2.12±00.00 + 0.00±00.00 41.65±00.28
Rust 1.008±0.000 1.88±00.00 + 0.00±00.00 42.93±00.33
D/ldc2 1.078±0.001 3.12±00.00 + 0.00±00.00 45.78±00.24
C++/g++ 1.097±0.002 3.38±00.00 + 0.00±00.00 46.21±00.59
C/gcc 1.112±0.001 1.38±00.12 + 0.00±00.00 47.10±00.09
D/gdc 1.116±0.001 6.75±00.00 + 0.00±00.00 48.89±00.28
C/clang 1.136±0.000 1.44±00.06 + 0.00±00.00 48.52±00.33
C++/clang++ 1.167±0.002 2.88±00.00 + 0.00±00.00 48.87±00.76
Nim/gcc 1.171±0.001 2.00±00.00 + 0.00±00.00 48.80±00.70
Java 1.191±0.000 41.74±00.09 + 1.00±00.00 49.32±00.60
Vala/gcc 1.202±0.001 5.25±00.12 + 0.00±00.00 50.70±00.44
Kotlin/JVM 1.224±0.002 44.48±00.07 + 1.31±00.25 51.60±00.64
Vala/clang 1.235±0.000 5.19±00.06 + 0.00±00.00 52.40±00.39
Go 1.256±0.000 3.75±00.00 + 0.00±00.00 52.69±00.80
C#/.NET Core 1.283±0.001 34.56±00.07 + 0.12±00.12 55.31±00.24
Go/gccgo 1.287±0.000 23.50±00.38 + 0.00±00.00 54.23±00.16
Zig 1.327±0.000 1.38±00.12 + 0.00±00.00 55.13±00.40
F#/.NET Core 1.507±0.003 39.33±00.10 + 0.19±00.06 65.29±00.40
Crystal 1.545±0.001 3.25±00.00 + 0.00±00.00 65.69±00.40
Julia 1.564±0.003 244.00±00.12 + 0.38±00.00 67.80±00.43
Nim/clang 1.579±0.000 2.25±00.00 + 0.00±00.00 65.59±00.57
OCaml 1.607±0.002 3.25±00.00 + 2.62±00.00 75.45±00.78
MLton 1.737±0.006 1.62±00.00 + 0.38±00.00 74.87±00.28
Chez Scheme 1.758±0.005 25.88±00.29 + 3.25±00.00 74.43±00.21
Racket 1.761±0.013 112.86±00.03 + 0.50±00.38 72.49±02.28
V/clang 1.846±0.009 2.06±00.06 + 0.00±00.00 80.00±00.53
Scala 2.379±0.003 66.03±00.14 + 196.44±00.12 108.88±00.92
Node.js 2.954±0.060 49.57±00.06 + 5.78±00.12 126.35±03.17
D/dmd 3.010±0.001 3.50±00.00 + 0.00±00.00 113.43±01.11
Haskell (MArray) 3.497±0.001 4.00±00.00 + 4.00±00.00 144.74±00.32
Haskell (FP) 3.819±0.008 4.12±00.06 + 4.06±00.06 163.47±01.56
Ruby/truffleruby (JVM) 4.645±0.070 354.81±05.59 + 478.37±54.20 221.41±05.66
Ruby/truffleruby 5.169±0.461 197.19±00.69 + 693.00±117.12 252.03±23.51
Swift 5.497±0.001 17.00±00.12 + 0.00±00.00 205.09±04.14
Lua/luajit 5.686±0.024 2.50±00.00 + 0.00±00.00 235.32±01.30
Python/pypy 9.398±0.038 60.19±00.19 + 30.10±00.05 419.65±03.72
Idris 15.088±0.032 21.55±00.26 + 8.00±00.00 677.49±01.79
Elixir 20.456±0.025 90.88±00.30 + 0.00±00.00 814.83±07.69
Ruby (--jit) 30.807±0.033 21.88±00.13 + 4.62±00.00 1275.90±04.28
PHP 34.055±0.041 17.61±00.06 + 0.00±00.00 1447.00±15.80
Lua 37.051±0.168 2.62±00.00 + 0.00±00.00 1528.23±05.85
Ruby/jruby 38.543±0.745 255.48±00.77 + 70.31±00.25 1732.28±34.61
Python 61.009±0.870 11.38±00.00 + 0.00±00.00 2601.10±34.54
Ruby 74.107±0.332 12.12±00.06 + 0.00±00.00 3209.85±28.74
Tcl (FP) 189.615±0.465 4.62±00.00 + 0.00±00.00 8409.77±69.18
Perl 225.517±1.873 7.06±00.06 + 0.12±00.00 9777.03±135.29
Tcl (OOP) 373.095±1.341 4.62±00.00 + 0.00±00.00 16604.13±190.89

mandel.b

Mandel in Brainfuck

Language Time, s Memory, MiB Energy, J
Scala (Staged) 7.118±0.079 228.10±04.74 + 71.23±06.06 356.09±07.48
C++/g++ 10.143±0.051 3.50±00.00 + 0.62±00.00 434.41±04.03
C/gcc 12.544±0.019 1.62±00.00 + 0.25±00.00 509.37±03.30
Vala/gcc 12.718±0.010 5.25±00.06 + 0.50±00.00 511.94±00.90
C#/.NET Core 12.742±0.028 34.69±00.06 + 1.25±00.06 494.75±01.32
C++/clang++ 12.803±0.022 3.00±00.00 + 0.62±00.00 545.25±01.09
Zig 13.030±0.017 1.50±00.00 + 1.00±00.00 556.10±03.71
Java 13.041±0.020 41.77±00.15 + 1.94±00.06 537.41±01.88
F#/.NET Core 13.104±0.049 39.32±00.12 + 2.00±00.00 527.94±02.27
C/clang 13.120±0.021 1.50±00.06 + 0.25±00.00 579.20±04.21
Kotlin/JVM 13.463±0.052 44.46±00.06 + 2.56±00.06 571.73±03.08
D/ldc2 14.340±0.012 3.06±00.06 + 0.75±00.00 591.01±03.53
Rust 14.460±0.021 1.94±00.06 + 0.25±00.00 577.37±02.52
Go 14.540±0.007 3.75±00.00 + 0.00±00.00 584.48±01.83
Vala/clang 14.802±0.012 5.12±00.00 + 0.50±00.00 595.99±01.53
Crystal 14.891±0.054 3.38±00.06 + 0.38±00.00 631.17±02.60
D/gdc 15.174±0.010 6.62±00.00 + 0.88±00.00 640.85±03.66
Racket (Staged) 15.722±0.085 99.73±00.12 + 71.69±01.06 613.58±05.37
Nim/gcc 15.766±0.016 2.00±00.06 + 1.50±00.00 635.49±04.84
Scala 15.785±0.067 65.94±00.09 + 145.00±00.19 692.62±02.98
Nim/clang 17.648±0.035 2.25±00.06 + 1.50±00.00 726.43±04.78
V/gcc 17.865±0.016 2.19±00.06 + 1.00±00.00 675.21±02.51
Swift 18.083±0.085 16.94±00.25 + 0.12±00.00 742.88±01.58
V/clang 18.691±0.126 2.25±00.00 + 1.00±00.00 813.69±05.16
Go/gccgo 19.206±0.179 23.38±00.12 + 0.00±00.00 796.93±09.15
OCaml 24.557±0.006 4.38±00.00 + 3.14±00.00 1195.26±09.14
Julia 30.315±0.060 244.06±00.07 + 0.25±00.00 1274.74±02.15
Chez Scheme 30.884±0.151 26.14±00.17 + 3.00±00.00 1367.28±07.09
Lua/luajit 31.804±0.026 2.50±00.00 + 0.25±00.00 1312.84±01.18
Node.js 34.692±0.097 49.51±00.00 + 8.19±00.06 1413.09±11.38
Racket 35.373±0.400 112.90±00.09 + 2.75±00.44 1573.97±10.34
MLton 35.680±0.051 1.62±00.00 + 4.28±00.00 1623.52±10.03
Haskell (MArray) 36.789±0.009 4.12±00.06 + 5.00±00.00 1522.12±01.36
D/dmd 38.105±0.005 3.50±00.00 + 0.75±00.00 1391.62±02.04
Python/pypy 41.153±0.229 60.12±00.19 + 31.00±00.09 1825.02±12.90
Ruby/truffleruby 48.381±0.372 196.62±01.12 + 582.69±01.75 2346.69±17.27
Ruby/truffleruby (JVM) 50.429±0.473 355.72±06.70 + 431.93±35.08 2253.46±91.61
Idris 64.504±0.237 21.73±00.14 + 9.88±00.00 2857.66±12.12
Haskell (FP) 79.505±0.192 4.00±00.00 + 74.81±00.06 3368.24±08.74

Base64

Testing base64 encoding/decoding of the large blob into the newly allocated buffers.

Base64

Language Time, s Memory, MiB Energy, J
PHP 0.065±0.001 18.04±00.18 + 0.00±00.00 3.27±00.03
C/clang (aklomp) 0.095±0.000 2.12±00.00 + 0.00±00.00 4.57±00.10
C/gcc (aklomp) 0.097±0.000 2.12±00.00 + 0.00±00.00 4.65±00.04
Go (base64x) 0.283±0.009 5.00±00.00 + 0.00±00.00 14.12±00.33
Node.js 0.635±0.005 47.63±00.06 + 81.13±00.95 29.16±00.53
Zig 0.703±0.000 1.81±00.06 + 0.12±00.00 26.42±00.09
Rust 0.842±0.000 2.31±00.06 + 0.12±00.00 34.91±00.90
C/clang 0.918±0.000 2.00±00.00 + 0.12±00.00 34.47±00.55
Nim/clang 0.945±0.000 2.75±00.00 + 5.00±00.00 38.18±01.09
Crystal 1.061±0.017 3.75±00.00 + 1.00±00.00 43.27±00.84
C/gcc 1.117±0.002 1.88±00.00 + 0.12±00.00 41.76±00.87
Nim/gcc 1.348±0.002 2.62±00.00 + 4.62±00.00 54.45±00.73
D/ldc2 1.352±0.002 3.75±00.00 + 3.38±00.00 55.63±00.63
Java 1.435±0.004 43.04±00.06 + 248.75±00.12 60.04±01.08
Kotlin/JVM 1.561±0.021 45.70±00.10 + 252.06±00.31 68.06±02.36
Go 1.592±0.007 4.31±00.06 + 0.00±00.00 66.55±01.37
Scala 1.594±0.050 63.26±00.10 + 315.25±04.19 68.76±01.29
V/gcc 1.641±0.003 2.75±00.00 + 2383.00±01.44 62.77±03.08
Vala/clang 1.646±0.003 5.62±00.00 + 0.12±00.00 64.21±01.67
Vala/gcc 1.647±0.002 5.75±00.00 + 0.12±00.00 65.43±02.11
Ruby 1.861±0.005 12.38±00.00 + 43.13±00.49 73.89±01.16
Perl (MIME::Base64) 1.997±0.008 14.88±00.00 + 0.00±00.00 81.05±01.87
Ruby (--jit) 2.033±0.005 46.26±00.04 + 64.14±00.04 79.75±00.49
C#/.NET Core 2.372±0.021 35.25±00.06 + 5.34±00.06 97.09±01.95
F#/.NET Core 2.445±0.076 39.86±00.06 + 20.22±00.94 98.47±03.45
Tcl 2.749±0.003 5.25±00.00 + 0.12±00.00 115.56±01.50
Go/gccgo 3.037±0.007 24.56±00.19 + 0.00±00.00 144.41±01.00
Julia 3.085±0.019 258.12±00.02 + 28.20±00.07 126.59±02.11
Python 3.099±0.002 11.25±00.00 + 0.10±00.00 126.46±01.27
V/clang 3.258±0.003 2.62±00.00 + 2388.00±01.12 121.78±02.22
Ruby/truffleruby (JVM) 3.505±0.036 350.81±05.12 + 256.81±12.70 182.88±06.01
Python/pypy 3.534±0.003 59.50±00.12 + 30.36±00.08 158.06±01.43
C++/clang++ (libcrypto) 3.664±0.007 6.38±00.00 + 0.25±00.00 152.34±02.81
C++/g++ (libcrypto) 3.664±0.003 7.00±00.00 + 0.25±00.00 153.36±02.88
D/dmd 3.760±0.003 3.62±00.00 + 3.38±00.00 158.04±01.48
Racket 3.886±0.007 95.64±00.10 + 21.81±00.19 156.59±01.21
D/gdc 3.976±0.003 7.00±00.06 + 3.62±00.00 166.06±02.47
Ruby/jruby 4.129±0.009 240.66±00.49 + 78.50±00.12 184.97±01.57
Ruby/truffleruby 8.789±0.034 196.12±01.06 + 544.31±06.06 422.24±01.81
Perl (MIME::Base64::Perl) 10.391±0.080 16.25±00.12 + 0.19±00.01 462.87±10.60

Json

Testing parsing and simple calculating of values from a big JSON file.

Few notes:

Json

Language Time, s Memory, MiB Energy, J
C++/g++ (simdjson On-Demand) 0.060±0.000 113.38±00.00 + 59.75±00.00 2.55±00.02
C++/clang++ (simdjson On-Demand) 0.062±0.000 113.00±00.00 + 59.75±00.00 2.59±00.03
C++/clang++ (DAW JSON Link NoCheck) 0.070±0.000 112.82±00.01 + 0.00±00.00 2.94±00.04
C++/clang++ (DAW JSON Link) 0.085±0.000 112.76±00.06 + 0.00±00.00 3.61±00.05
C++/g++ (DAW JSON Link NoCheck) 0.087±0.000 113.21±00.00 + 0.00±00.00 3.54±00.05
C++/g++ (DAW JSON Link) 0.089±0.000 113.21±00.00 + 0.00±00.00 3.66±00.02
Rust (Serde Custom) 0.103±0.000 111.62±00.00 + 0.00±00.00 4.43±00.03
C++/clang++ (simdjson DOM) 0.103±0.001 112.88±00.00 + 175.06±00.69 4.72±00.08
C++/g++ (simdjson DOM) 0.107±0.002 113.44±00.06 + 175.86±00.89 4.89±00.06
Rust (Serde Typed) 0.110±0.000 111.75±00.00 + 11.12±00.00 4.70±00.02
C++/g++ (gason) 0.127±0.000 113.21±00.00 + 96.75±00.00 5.16±00.06
C++/clang++ (gason) 0.133±0.001 112.70±00.00 + 96.75±00.00 5.35±00.04
D/ldc2 (Mir Asdf DOM) 0.133±0.001 112.81±00.06 + 61.25±00.00 5.56±00.07
Scala (jsoniter-scala) 0.147±0.001 283.94±00.09 + 25.62±00.19 7.89±00.03
C++/g++ (RapidJSON) 0.154±0.001 113.21±00.00 + 123.44±00.88 6.61±00.07
C++/clang++ (RapidJSON) 0.194±0.001 112.95±00.00 + 128.62±00.00 8.25±00.16
Go (rjson custom) 0.199±0.000 114.00±00.06 + 0.00±00.00 7.68±00.01
Go (Sonic) 0.209±0.001 121.88±00.12 + 0.00±00.00 9.14±00.05
C++/g++ (RapidJSON Precise) 0.212±0.001 113.21±00.00 + 128.75±00.00 9.15±00.11
D/ldc2 (Mir Amazon's Ion DOM) 0.220±0.001 113.00±00.00 + 80.75±00.00 9.29±00.05
Go (rjson) 0.224±0.000 114.00±00.00 + 0.00±00.00 8.65±00.03
Zig 0.239±0.001 111.12±00.00 + 36.44±00.06 10.29±00.04
Go (goccy/go-json) 0.252±0.000 114.50±00.06 + 0.00±00.00 9.99±00.08
C++/clang++ (RapidJSON Precise) 0.262±0.001 112.83±00.00 + 128.75±00.00 11.08±00.14
C/gcc (yajl) 0.362±0.000 111.25±00.00 + 0.00±00.00 15.71±00.12
C/clang (yajl) 0.364±0.000 111.25±00.00 + 0.00±00.00 15.65±00.08
C++/g++ (RapidJSON SAX) 0.366±0.001 113.33±00.00 + 0.00±00.00 15.95±00.12
C++/g++ (Boost.JSON) 0.374±0.002 113.47±00.00 + 307.75±00.00 15.97±00.15
Node.js 0.374±0.001 158.06±00.06 + 283.88±03.00 18.22±00.13
C++/clang++ (Boost.JSON) 0.377±0.002 113.00±00.07 + 307.75±00.00 16.12±00.12
Nim/clang (jsony) 0.386±0.001 112.12±00.00 + 159.75±01.94 16.19±00.20
Nim/gcc (jsony) 0.418±0.001 111.75±00.00 + 155.62±01.50 17.56±00.10
C++/clang++ (RapidJSON SAX) 0.423±0.001 195.20±00.00 + 0.00±00.00 17.99±00.12
C++/g++ (RapidJSON SAX Precise) 0.433±0.000 113.34±00.00 + 0.12±00.00 19.54±00.20
Go (jsoniter) 0.482±0.001 114.50±00.12 + 0.00±00.00 19.93±00.04
Crystal (Pull) 0.484±0.002 113.44±00.06 + 18.00±00.00 21.76±00.28
C++/clang++ (RapidJSON SAX Precise) 0.500±0.001 195.20±00.00 + 0.00±00.00 22.35±00.40
C#/.NET Core (System.Text.Json) 0.500±0.001 481.12±00.00 + 119.63±00.08 24.55±00.12
Crystal (Schema) 0.510±0.001 113.38±00.00 + 48.31±00.06 22.61±00.39
Rust (Serde Untyped) 0.557±0.002 111.88±00.12 + 839.75±00.00 23.75±00.42
Java (DSL-JSON) 0.571±0.020 277.81±00.14 + 163.69±00.38 29.34±01.43
Python/pypy 0.606±0.001 280.66±00.07 + 125.44±00.01 26.55±00.11
V/clang 0.635±0.001 112.12±00.00 + 496.00±00.00 27.08±00.57
V/gcc 0.639±0.005 111.88±00.00 + 496.00±00.00 27.16±00.35
Nim/clang (Packedjson) 0.664±0.002 112.38±00.00 + 294.25±00.00 28.59±00.36
Nim/gcc (Packedjson) 0.674±0.004 112.12±00.00 + 294.12±00.00 28.60±00.31
Julia (JSON3) 0.700±0.001 631.76±00.04 + 332.81±01.00 29.33±00.22
CPython (UltraJSON) 0.700±0.002 123.35±00.12 + 478.56±01.81 28.00±00.40
Perl (Cpanel::JSON::XS) 0.767±0.011 125.38±00.00 + 402.88±00.00 32.13±00.63
Crystal 0.795±0.007 113.44±00.06 + 392.12±00.00 34.30±00.39
Go 0.834±0.001 114.00±00.06 + 0.00±00.00 34.46±00.07
PHP 0.836±0.001 127.36±00.12 + 517.82±00.00 36.28±00.23
Python 0.936±0.003 121.30±00.06 + 325.88±00.00 39.40±00.44
Ruby (--jit) 1.014±0.001 126.75±00.02 + 210.64±00.03 42.62±00.35
Ruby 1.031±0.002 122.00±00.12 + 212.08±00.20 43.59±00.44
Nim/gcc 1.036±0.005 112.12±00.00 + 1000.88±00.38 43.26±00.27
Nim/clang 1.092±0.004 112.38±00.00 + 999.00±00.00 45.82±00.27
Clojure 1.128±0.017 476.94±02.22 + 405.56±14.12 59.15±01.15
C#/.NET Core 1.169±0.005 488.06±00.06 + 41.46±00.17 57.40±00.51
C++/clang++ (Nlohmann) 1.182±0.002 112.95±00.00 + 359.88±00.00 50.72±00.62
C++/clang++ (json-c) 1.185±0.002 112.96±00.00 + 1215.88±00.00 49.83±00.62
C++/g++ (json-c) 1.187±0.004 113.35±00.00 + 1215.88±00.00 49.45±00.40
Go/gccgo 1.256±0.003 139.25±00.12 + 0.00±00.00 51.58±00.27
C++/g++ (Nlohmann) 1.266±0.003 113.33±00.00 + 447.88±00.00 53.77±00.41
F#/.NET Core (System.Text.Json) 1.494±0.003 490.33±00.11 + 133.64±00.43 70.35±00.46
Ruby (YAJL) 1.565±0.019 121.69±00.06 + 233.88±00.00 68.39±01.94
D/ldc2 1.633±0.002 113.12±00.06 + 638.62±00.00 70.66±00.51
C3 1.959±0.003 111.88±00.00 + 795.88±00.00 82.11±00.61
Haskell 2.019±0.013 115.75±00.12 + 723.44±00.12 86.17±00.72
C++/clang++ (Boost.PropertyTree) 2.589±0.004 195.33±00.00 + 1232.50±00.00 111.50±00.48
Rust (jq) 2.637±0.006 113.50±00.00 + 905.97±02.12 110.08±00.97
D/dmd 2.778±0.003 113.50±00.00 + 626.00±00.00 121.50±00.76
Ruby/jruby 2.838±0.016 555.29±00.73 + 572.50±77.88 152.07±00.83
C++/g++ (Boost.PropertyTree) 2.907±0.010 113.46±00.00 + 1439.88±00.00 124.59±00.96
Vala/gcc 2.980±0.017 115.25±00.00 + 980.00±00.00 127.79±01.75
Vala/clang 2.983±0.012 115.25±00.00 + 980.00±00.00 127.36±00.59
Odin 3.026±0.007 111.38±00.00 + 20.00±00.00 126.12±01.22
D/gdc 3.459±0.007 116.75±00.00 + 708.88±00.06 148.15±01.48
Racket 3.861±0.015 223.42±00.33 + 288.29±04.07 163.00±01.74
Perl (JSON::Tiny) 9.034±0.089 126.00±00.00 + 528.56±00.00 401.98±06.07
Ruby/truffleruby (JVM) 10.618±0.148 482.80±07.29 + 2212.51±207.00 678.79±08.84
Ruby/truffleruby 11.341±0.066 402.19±11.19 + 1941.56±30.56 665.04±03.89

Matmul

Testing allocating and multiplying matrices.

Matmul

Language Time, s Memory, MiB Energy, J
D/ldc2 (lubeck) 0.042±0.001 16.38±01.31 + 47.88±01.38 4.27±00.05
Python (NumPy) 0.066±0.001 32.78±02.50 + 52.17±02.29 5.62±00.07
Nim/gcc (Arraymancer) 0.073±0.005 5.62±00.12 + 58.06±00.19 5.88±00.23
C++/g++ (Eigen) 0.081±0.000 10.06±01.62 + 79.75±01.75 5.68±00.02
C++/clang++ (Eigen) 0.081±0.000 12.88±00.50 + 77.46±00.56 6.41±00.04
Java (ND4J) 0.082±0.001 122.16±01.44 + 92.56±00.19 6.51±00.11
Rust (ndarray) 0.091±0.001 2.62±00.00 + 68.36±00.00 6.20±00.10
Nim/clang (Arraymancer) 0.114±0.045 18.81±00.44 + 45.62±00.50 8.31±02.57
Julia (threads: 2) 0.120±0.000 259.86±00.08 + 56.95±00.02 6.69±00.05
Julia (threads: 1) 0.170±0.000 259.82±00.17 + 56.36±00.04 8.09±00.03
V/clang (VSL) 0.224±0.002 10.62±00.12 + 51.60±00.03 18.12±00.11
V/clang (VSL + CBLAS) 0.227±0.001 10.62±00.12 + 51.47±00.09 18.15±00.23
V/gcc (VSL + CBLAS) 0.423±0.003 2.50±00.00 + 51.38±00.00 32.87±00.19
V/gcc (VSL) 0.441±0.002 2.50±00.00 + 51.38±00.00 30.29±00.18
Julia (no BLAS) 1.113±0.017 256.38±00.00 + 51.50±00.00 49.41±00.63
D/gdc 1.505±0.001 7.12±00.00 + 4.00±00.00 58.37±00.11
D/ldc2 1.722±0.001 3.62±00.00 + 70.38±00.00 64.31±00.15
D/dmd 1.880±0.002 3.56±00.06 + 70.38±00.00 72.70±00.56
C/gcc 3.034±0.001 2.00±00.00 + 68.38±00.00 113.24±00.33
V/gcc 3.036±0.000 2.50±00.06 + 68.75±00.00 113.56±00.46
Java 3.057±0.001 43.14±00.13 + 68.75±00.12 123.21±00.95
Vala/clang 3.062±0.001 5.54±00.04 + 68.38±00.00 109.60±00.81
V/clang 3.065±0.000 2.88±00.00 + 68.75±00.00 109.23±00.06
C/clang 3.067±0.001 2.00±00.00 + 68.38±00.00 109.34±01.23
Rust 3.067±0.001 2.25±00.12 + 68.50±00.00 109.32±00.64
Zig 3.071±0.003 1.88±00.00 + 68.56±00.06 109.78±00.28
Nim/gcc 3.091±0.002 2.62±00.06 + 57.88±00.00 115.31±00.37
Swift 3.102±0.001 8.38±00.12 + 68.62±00.00 113.76±00.36
Nim/clang 3.121±0.001 2.94±00.06 + 57.75±00.00 111.65±00.36
Vala/gcc 3.131±0.001 5.45±00.06 + 68.50±00.00 118.43±00.81
Go 3.153±0.001 4.12±00.12 + 0.00±00.00 115.20±00.44
Crystal 3.154±0.001 3.75±00.00 + 59.50±00.00 116.10±00.24
Go/gccgo 3.159±0.001 24.44±00.12 + 0.00±00.00 115.54±00.39
Kotlin/JVM 3.160±0.002 43.80±00.16 + 70.00±00.12 124.32±00.30
Node.js 3.212±0.005 57.91±00.28 + 70.83±00.19 126.71±00.50
Scala 3.243±0.003 64.43±00.14 + 158.69±00.06 119.81±00.30
Python/pypy 3.263±0.002 60.62±00.12 + 68.94±00.06 136.66±00.49
C#/.NET Core 4.725±0.001 36.62±00.06 + 69.44±00.06 193.26±01.10
Ruby/truffleruby (JVM) 17.667±0.235 417.21±09.41 + 339.96±72.03 722.45±10.75
Ruby/truffleruby 17.776±0.189 324.88±03.31 + 504.50±13.19 638.49±08.58
Python 136.267±0.437 11.62±00.06 + 68.75±00.00 6131.10±41.35
Perl 149.049±1.633 8.62±00.06 + 379.75±00.00 6511.38±65.17
Ruby (--jit) 152.991±0.294 21.83±01.04 + 68.62±00.06 6802.85±30.51
Tcl 205.116±0.537 7.62±00.00 + 400.38±00.00 9417.77±28.45
Ruby 211.868±0.274 12.75±00.00 + 69.62±00.00 9430.93±36.77
Ruby/jruby 330.203±16.724 299.99±00.72 + 880.65±189.19 13544.82±520.08

Primes

Testing:

  • generating primes using the optimized sieve of Atkin;
  • prefix search for their decimal numbers using Trie data structure.

Notes:

  • All languages but V and Python use unordered hashmaps (V and Python don't provide those out of box, and their hashmaps use keys in the insertion order);
  • The results are always sorted (could be unstable or stable though).

Primes

Language Time, s Memory, MiB Energy, J
C++/g++ 0.064±0.000 3.62±00.00 + 85.55±00.44 2.57±00.06
Go 0.070±0.000 3.62±00.00 + 0.00±00.00 3.09±00.09
Zig 0.071±0.000 1.50±00.12 + 64.69±00.06 2.94±00.01
C++/clang++ 0.075±0.000 3.25±00.00 + 64.48±00.19 2.90±00.06
V/clang 0.100±0.000 2.12±00.00 + 200.44±00.69 4.18±00.08
Rust 0.102±0.000 2.00±00.00 + 74.25±00.12 4.05±00.06
V/gcc 0.106±0.001 2.25±00.00 + 211.06±02.06 4.52±00.05
Java 0.113±0.000 41.79±00.14 + 107.88±03.81 6.06±00.22
Crystal 0.144±0.000 3.50±00.00 + 89.88±00.00 5.91±00.10
Scala 0.184±0.004 66.51±00.15 + 158.44±02.00 10.67±00.30
Node.js 0.216±0.001 46.88±00.06 + 227.55±04.77 10.68±00.15
Nim/gcc 0.293±0.000 1.75±00.12 + 582.06±01.00 11.25±00.17
Nim/clang 0.295±0.001 2.00±00.00 + 603.44±05.06 11.62±00.12
Lua/luajit 0.305±0.001 2.50±00.00 + 157.22±01.05 12.36±00.17
Julia 0.419±0.002 258.38±00.12 + 215.71±03.03 16.24±00.31
Python/pypy 0.617±0.003 59.62±00.12 + 247.16±00.04 25.09±00.34
Racket 0.757±0.004 109.51±00.71 + 248.35±01.07 30.05±00.58
Ruby/truffleruby 0.910±0.010 197.81±01.44 + 672.44±54.69 61.70±00.61
Lua 1.230±0.005 2.56±00.06 + 283.84±00.75 50.66±00.69
Ruby (--jit) 1.289±0.014 23.96±00.03 + 174.84±00.31 54.03±01.54
Ruby/truffleruby (JVM) 1.363±0.033 350.78±04.33 + 506.71±28.12 88.50±02.14
Ruby/jruby 1.491±0.021 250.45±00.62 + 577.06±02.31 80.03±01.19
Ruby 1.992±0.027 12.12±00.00 + 183.69±00.50 85.16±02.19
Python 2.096±0.012 11.44±00.06 + 181.46±01.00 90.37±01.30

Tests Execution

Environment

CPU: Intel(R) Xeon(R) E-2324G

Base Docker image: Debian GNU/Linux trixie/sid

Language Version
.NET Core 9.0.304
C#/.NET Core 4.14.0-3.25359.3 (6dbcfd2f)
C3 0.7.4
Chez Scheme 10.0.0
Clojure "1.12.1"
Crystal 1.17.1
D/dmd v2.111.0
D/gdc 14.2.0
D/ldc2 1.41.0
Elixir 1.18.3
F#/.NET Core 13.9.303.0 for F# 9.0
Go go1.25.0
Go/gccgo 14.2.0
Haskell 9.10.2
Idris 2 0.7.0
Java 24.0.2
Julia v"1.11.6"
Kotlin 2.2.0
Lua 5.4.7
Lua/luajit 2.1.1748495995
MLton 20241230
Nim 2.2.4
Node.js v24.5.0
OCaml 5.3.0
Odin dev-2025-08-nightly
PHP 8.4.11
Perl v5.40.1
Python 3.13.5
Python/pypy 7.3.20-final0 for Python 3.11.13
Racket "8.17"
Ruby 3.4.5p51
Ruby/jruby 10.0.2.0
Ruby/truffleruby 24.2.2
Rust 1.89.0
Scala 3.7.2
Swift 6.1.2
Tcl 8.6
V 0.4.11 b9e5757
Vala 0.56.18
Zig 0.14.1
clang/clang++ 19.1.7 (3+b1)
gcc/g++ 14.2.0

Using Docker

Build the image:

$ docker build docker/ -t benchmarks

Run the image:

$ docker run -it --rm -v $(pwd):/src benchmarks <cmd>

where <cmd> is:

  • versions (print installed language versions);
  • shell (start the shell);
  • brainfuck bench (build and run Brainfuck bench.b benchmarks);
  • brainfuck mandel (build and run Brainfuck mandel.b benchmarks);
  • base64 (build and run Base64 benchmarks);
  • json (build and run Json benchmarks);
  • matmul (build and run Matmul benchmarks);
  • primes (build and run Primes benchmarks);

Please note that the actual measurements provided in the project are taken semi-manually (via shell) as the full update takes days and could have occassional issues in Docker.

There is a ./run.sh that could be used to simplify Docker usage:

  • ./run.sh build (build the image);
  • ./run.sh make versions (run the image with the versions command);
  • sudo ./run.sh shell (run the image with the `shell' command, sudo is required to read energy levels).

Manual Execution

Makefiles contain recipes for building and executing tests with the proper dependencies. Please use make run (and make run2 where applicable). The measurements are taken using analyze.rb script:

$ cd <test suite>
$ ../analyze.rb make run
$ ../analyze.rb make run[<single test>]

Please note that the measurements could take hours. It uses 10 iterations by default, but it could be changed using ATTEMPTS environment variable:

$ ATTEMPTS=1 ../analyze.rb make run

Prerequisites

Please use Dockerfile as a reference regarding which packages and tools are required.

For all (optional):

  • Powercap for reading energy counters in Linux (Debian package powercap-utils).

For Python:

  • NumPy for matmul tests (Debian package python3-numpy).
  • UltraJSON for JSON tests (Debian package python3-ujson).

For C++:

  • Boost for JSON tests (Debian package libboost-dev).
  • JSON-C for JSON tests (Debian package libjson-c-dev).

For Rust:

  • libjq for jq test (Debian packages libjq-dev, libonig-dev and environment variable JQ_LIB_DIR=/usr/lib/x86_64-linux-gnu/).

For Java, Scala:

  • Coursier for downloading Maven artifacts.

For Haskell:

  • network for TCP connectivity between the tests and the test runner.
  • raw-strings-qq for raw string literals used in tests.

For Perl:

  • cpanminus for installing modules from CPAN (Debian package cpanminus).

For Vala:

  • JSON-GLib for JSON tests (Debian package libjson-glib-dev).

Contribution

Please follow the criteria specified in the overview. Besides that please ensure that the communication protocol between a test and the test runner is satisfied:

  • The test runner listens on localhost:9001;
  • All messages are sent using TCP sockets closed immediately after the message has been sent;
  • There are two messages sent from a test (it establishes the measurement boundary):
    1. The beginning message having the format name of the test/tprocess ID (the process ID is used to measure the memory consumption). Please note that the name of the test couldn't use Tab character as it's a delimiter;
    2. The end message with any content (mostly it's "stop" for consistency).
  • The test runner could be unavailable (if the test is launched as is) and the test should gracefully handle it.

Makefile guide

Binary executables

If the test is compiled into a single binary, then two sections of the Makefile require changes:

  • append a new target (the final binary location) into executables variable;
  • append the proper target rule.

Compiled artifacts

If the test is compiled, but can't be executed directly as a binary, then three sections of the Makefile require changes:

  • append a new target (the final artifact location) into artifacts variable;
  • append the proper target rule to compile the test;
  • append run[<target_artifact>] rule to run the test.

Scripting language

If the test doesn't require compilation, then two sections of the Makefile requires changes:

  • append run[<script_file>] into all_runners variable;
  • append run[<script_file>] rule to run the test.

README update

TOC is regenerated using git-markdown-toc:

./run.sh toc

About

Some benchmarks of different languages

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published