Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
test: State benchmarks (FuelLabs#1226)
Related issues: - Ref FuelLabs#1134 These benches demonstrate the performance of: - Inserting 1 key/value pair into the RocksDB table that contains `n` state entries for a single contracts. `n = [0, 1, 10, 100, 1,000, 10,000, 100,000, 1,000,000]` - Inserting 1 key/value pair into the RocksDB table that contains 1 state entries with `n` existing contracts. `n = [0, 1, 10, 100, 1,000, 10,000, 100,000, 1,000,000]` Bench results: ``` state single contract/insert state with 0 preexisting entries time: [11.991 µs 12.132 µs 12.310 µs] change: [-4.7172% -1.6523% +1.0884%] (p = 0.29 > 0.05) No change in performance detected. Found 9 outliers among 100 measurements (9.00%) 5 (5.00%) high mild 4 (4.00%) high severe state single contract/insert state with 1 preexisting entries time: [25.009 µs 28.694 µs 32.962 µs] Found 6 outliers among 100 measurements (6.00%) 5 (5.00%) high mild 1 (1.00%) high severe state single contract/insert state with 10 preexisting entries time: [39.002 µs 41.617 µs 44.324 µs] Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high severe state single contract/insert state with 100 preexisting entries time: [60.968 µs 63.985 µs 67.259 µs] Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe state single contract/insert state with 1,000 preexisting entries time: [87.465 µs 91.207 µs 95.394 µs] change: [+0.4239% +4.5684% +8.9542%] (p = 0.04 < 0.05) Change within noise threshold. Found 7 outliers among 100 measurements (7.00%) 6 (6.00%) high mild 1 (1.00%) high severe state single contract/insert state with 10,000 preexisting entries time: [117.22 µs 121.38 µs 125.93 µs] Found 6 outliers among 100 measurements (6.00%) 6 (6.00%) high mild state single contract/insert state with 100,000 preexisting entries time: [145.05 µs 148.30 µs 151.60 µs] change: [-3.4471% -0.8145% +1.9661%] (p = 0.56 > 0.05) No change in performance detected. state single contract/insert state with 1,000,000 preexisting entries time: [186.36 µs 190.05 µs 193.80 µs] Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high mild state multiple contracts/insert state with 0 preexisting entries time: [11.957 µs 12.080 µs 12.237 µs] change: [-19.431% -7.6642% +0.8636%] (p = 0.27 > 0.05) No change in performance detected. Found 9 outliers among 100 measurements (9.00%) 5 (5.00%) high mild 4 (4.00%) high severe state multiple contracts/insert state with 1 preexisting entries time: [11.872 µs 12.008 µs 12.199 µs] Found 5 outliers among 100 measurements (5.00%) 2 (2.00%) high mild 3 (3.00%) high severe state multiple contracts/insert state with 10 preexisting entries time: [12.189 µs 12.252 µs 12.320 µs] Found 4 outliers among 100 measurements (4.00%) 1 (1.00%) high mild 3 (3.00%) high severe state multiple contracts/insert state with 100 preexisting entries time: [12.538 µs 12.691 µs 12.867 µs] Found 8 outliers among 100 measurements (8.00%) 5 (5.00%) high mild 3 (3.00%) high severe state multiple contracts/insert state with 1,000 preexisting entries time: [13.041 µs 13.278 µs 13.640 µs] change: [-1.9317% +1.1895% +4.0288%] (p = 0.46 > 0.05) No change in performance detected. Found 2 outliers among 100 measurements (2.00%) 2 (2.00%) high severe state multiple contracts/insert state with 10,000 preexisting entries time: [13.350 µs 13.529 µs 13.729 µs] Found 3 outliers among 100 measurements (3.00%) 2 (2.00%) high mild 1 (1.00%) high severe state multiple contracts/insert state with 100,000 preexisting entries time: [13.973 µs 14.312 µs 14.719 µs] change: [+4.3928% +9.0688% +14.995%] (p = 0.00 < 0.05) Performance has regressed. Found 6 outliers among 100 measurements (6.00%) 2 (2.00%) high mild 4 (4.00%) high severe state multiple contracts/insert state with 1,000,000 preexisting entries time: [13.891 µs 14.096 µs 14.314 µs] Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe ``` **Conclusion:** For inserting `n` state entries for the same contract, the performance appears to be logarithmic with `n`. For inserting 1 state entry for a contract, with `n` existing contracts, the performance appears to be fairly constant, i.e., the number of existing contracts does not have an impact on the performance of inserting state for a contract. Note that these benchmarks were taken after the change to use RocksDB in benches (FuelLabs#1227).
- Loading branch information