Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: use availableParallelism to determine thread pool size #5691

Merged
merged 1 commit into from
Jun 23, 2023

Conversation

nflaig
Copy link
Member

@nflaig nflaig commented Jun 23, 2023

Motivation

From node docs

os.cpus().length should not be used to calculate the amount of parallelism available to an application. Use os.availableParallelism() for this purpose.

Description

Use os.availableParallelism() instead of os.cpus().length to determine thread pool size.

This is supported from node v18.14.0, see availableParallelism.

@nflaig nflaig requested a review from a team as a code owner June 23, 2023 18:28
@wemeetagain wemeetagain enabled auto-merge (squash) June 23, 2023 18:32
@github-actions
Copy link
Contributor

github-actions bot commented Jun 23, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 3b18a8a Previous: 85ac6cb Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 532.23 us/op 955.45 us/op 0.56
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 49.311 us/op 48.363 us/op 1.02
BLS verify - blst-native 1.2450 ms/op 1.2212 ms/op 1.02
BLS verifyMultipleSignatures 3 - blst-native 2.5231 ms/op 2.4977 ms/op 1.01
BLS verifyMultipleSignatures 8 - blst-native 5.3262 ms/op 5.3344 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 19.258 ms/op 19.205 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 26.373 us/op 25.807 us/op 1.02
BLS aggregatePubkeys 128 - blst-native 101.40 us/op 101.13 us/op 1.00
getAttestationsForBlock 58.087 ms/op 59.855 ms/op 0.97
isKnown best case - 1 super set check 286.00 ns/op 257.00 ns/op 1.11
isKnown normal case - 2 super set checks 268.00 ns/op 245.00 ns/op 1.09
isKnown worse case - 16 super set checks 272.00 ns/op 246.00 ns/op 1.11
CheckpointStateCache - add get delete 5.1380 us/op 5.5580 us/op 0.92
validate gossip signedAggregateAndProof - struct 2.8051 ms/op 2.7835 ms/op 1.01
validate gossip attestation - struct 1.3432 ms/op 1.3209 ms/op 1.02
pickEth1Vote - no votes 1.3324 ms/op 1.3664 ms/op 0.98
pickEth1Vote - max votes 12.199 ms/op 12.655 ms/op 0.96
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.9072 ms/op 9.8872 ms/op 0.90
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 16.440 ms/op 16.733 ms/op 0.98
pickEth1Vote - Eth1Data fastSerialize value x2048 655.40 us/op 675.57 us/op 0.97
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.9204 ms/op 5.0638 ms/op 1.17
bytes32 toHexString 499.00 ns/op 509.00 ns/op 0.98
bytes32 Buffer.toString(hex) 345.00 ns/op 407.00 ns/op 0.85
bytes32 Buffer.toString(hex) from Uint8Array 555.00 ns/op 609.00 ns/op 0.91
bytes32 Buffer.toString(hex) + 0x 361.00 ns/op 419.00 ns/op 0.86
Object access 1 prop 0.17200 ns/op 0.17300 ns/op 0.99
Map access 1 prop 0.16600 ns/op 0.16500 ns/op 1.01
Object get x1000 6.5710 ns/op 6.5630 ns/op 1.00
Map get x1000 0.63500 ns/op 0.57100 ns/op 1.11
Object set x1000 54.155 ns/op 61.766 ns/op 0.88
Map set x1000 46.211 ns/op 46.339 ns/op 1.00
Return object 10000 times 0.24200 ns/op 0.24120 ns/op 1.00
Throw Error 10000 times 4.2321 us/op 4.2998 us/op 0.98
fastMsgIdFn sha256 / 200 bytes 3.4870 us/op 3.6190 us/op 0.96
fastMsgIdFn h32 xxhash / 200 bytes 281.00 ns/op 325.00 ns/op 0.86
fastMsgIdFn h64 xxhash / 200 bytes 394.00 ns/op 464.00 ns/op 0.85
fastMsgIdFn sha256 / 1000 bytes 11.544 us/op 11.688 us/op 0.99
fastMsgIdFn h32 xxhash / 1000 bytes 405.00 ns/op 447.00 ns/op 0.91
fastMsgIdFn h64 xxhash / 1000 bytes 460.00 ns/op 540.00 ns/op 0.85
fastMsgIdFn sha256 / 10000 bytes 104.20 us/op 103.82 us/op 1.00
fastMsgIdFn h32 xxhash / 10000 bytes 1.9180 us/op 1.9980 us/op 0.96
fastMsgIdFn h64 xxhash / 10000 bytes 1.3450 us/op 1.4510 us/op 0.93
enrSubnets - fastDeserialize 64 bits 1.3930 us/op 1.4860 us/op 0.94
enrSubnets - ssz BitVector 64 bits 486.00 ns/op 599.00 ns/op 0.81
enrSubnets - fastDeserialize 4 bits 177.00 ns/op 184.00 ns/op 0.96
enrSubnets - ssz BitVector 4 bits 479.00 ns/op 536.00 ns/op 0.89
prioritizePeers score -10:0 att 32-0.1 sync 2-0 105.78 us/op 108.65 us/op 0.97
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 131.87 us/op 155.50 us/op 0.85
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 174.53 us/op 191.65 us/op 0.91
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 314.43 us/op 369.11 us/op 0.85
prioritizePeers score 0:0 att 64-1 sync 4-1 369.39 us/op 424.47 us/op 0.87
array of 16000 items push then shift 1.7284 us/op 1.6704 us/op 1.03
LinkedList of 16000 items push then shift 9.0080 ns/op 9.1390 ns/op 0.99
array of 16000 items push then pop 113.25 ns/op 113.22 ns/op 1.00
LinkedList of 16000 items push then pop 10.247 ns/op 9.2060 ns/op 1.11
array of 24000 items push then shift 2.4400 us/op 2.3861 us/op 1.02
LinkedList of 24000 items push then shift 9.2740 ns/op 9.0680 ns/op 1.02
array of 24000 items push then pop 89.422 ns/op 82.397 ns/op 1.09
LinkedList of 24000 items push then pop 9.3790 ns/op 8.8150 ns/op 1.06
intersect bitArray bitLen 8 14.335 ns/op 13.520 ns/op 1.06
intersect array and set length 8 109.64 ns/op 102.35 ns/op 1.07
intersect bitArray bitLen 128 53.686 ns/op 45.363 ns/op 1.18
intersect array and set length 128 1.4288 us/op 1.2242 us/op 1.17
Buffer.concat 32 items 3.1990 us/op 3.0240 us/op 1.06
Uint8Array.set 32 items 3.0910 us/op 2.8640 us/op 1.08
transfer serialized Status (84 B) 2.2710 us/op 2.2610 us/op 1.00
copy serialized Status (84 B) 2.0630 us/op 1.9640 us/op 1.05
transfer serialized SignedVoluntaryExit (112 B) 2.3820 us/op 2.3860 us/op 1.00
copy serialized SignedVoluntaryExit (112 B) 1.8230 us/op 1.9200 us/op 0.95
transfer serialized ProposerSlashing (416 B) 3.2130 us/op 2.5500 us/op 1.26
copy serialized ProposerSlashing (416 B) 2.9220 us/op 3.0260 us/op 0.97
transfer serialized Attestation (485 B) 3.0150 us/op 2.9940 us/op 1.01
copy serialized Attestation (485 B) 2.7580 us/op 3.1520 us/op 0.88
transfer serialized AttesterSlashing (33232 B) 2.8150 us/op 2.9040 us/op 0.97
copy serialized AttesterSlashing (33232 B) 10.978 us/op 6.8980 us/op 1.59
transfer serialized Small SignedBeaconBlock (128000 B) 3.4930 us/op 3.3520 us/op 1.04
copy serialized Small SignedBeaconBlock (128000 B) 27.170 us/op 15.179 us/op 1.79
transfer serialized Avg SignedBeaconBlock (200000 B) 3.1760 us/op 3.6820 us/op 0.86
copy serialized Avg SignedBeaconBlock (200000 B) 41.444 us/op 23.008 us/op 1.80
transfer serialized BlobsSidecar (524380 B) 4.0680 us/op 3.8120 us/op 1.07
copy serialized BlobsSidecar (524380 B) 243.85 us/op 115.45 us/op 2.11
transfer serialized Big SignedBeaconBlock (1000000 B) 3.9650 us/op 3.9070 us/op 1.01
copy serialized Big SignedBeaconBlock (1000000 B) 360.99 us/op 277.03 us/op 1.30
pass gossip attestations to forkchoice per slot 3.2702 ms/op 2.6884 ms/op 1.22
forkChoice updateHead vc 100000 bc 64 eq 0 2.4466 ms/op 2.0787 ms/op 1.18
forkChoice updateHead vc 600000 bc 64 eq 0 11.747 ms/op 12.237 ms/op 0.96
forkChoice updateHead vc 1000000 bc 64 eq 0 19.695 ms/op 22.203 ms/op 0.89
forkChoice updateHead vc 600000 bc 320 eq 0 17.252 ms/op 17.233 ms/op 1.00
forkChoice updateHead vc 600000 bc 1200 eq 0 98.271 ms/op 84.946 ms/op 1.16
forkChoice updateHead vc 600000 bc 64 eq 1000 22.708 ms/op 20.552 ms/op 1.10
forkChoice updateHead vc 600000 bc 64 eq 10000 23.987 ms/op 22.682 ms/op 1.06
forkChoice updateHead vc 600000 bc 64 eq 300000 32.361 ms/op 31.587 ms/op 1.02
computeDeltas 3.6083 ms/op 3.3085 ms/op 1.09
computeProposerBoostScoreFromBalances 1.9739 ms/op 1.8545 ms/op 1.06
altair processAttestation - 250000 vs - 7PWei normalcase 4.0915 ms/op 2.5597 ms/op 1.60
altair processAttestation - 250000 vs - 7PWei worstcase 5.5224 ms/op 4.3390 ms/op 1.27
altair processAttestation - setStatus - 1/6 committees join 150.20 us/op 140.91 us/op 1.07
altair processAttestation - setStatus - 1/3 committees join 291.45 us/op 275.84 us/op 1.06
altair processAttestation - setStatus - 1/2 committees join 397.38 us/op 369.85 us/op 1.07
altair processAttestation - setStatus - 2/3 committees join 500.00 us/op 462.22 us/op 1.08
altair processAttestation - setStatus - 4/5 committees join 706.27 us/op 657.40 us/op 1.07
altair processAttestation - setStatus - 100% committees join 820.44 us/op 779.09 us/op 1.05
altair processBlock - 250000 vs - 7PWei normalcase 21.726 ms/op 19.139 ms/op 1.14
altair processBlock - 250000 vs - 7PWei normalcase hashState 33.194 ms/op 27.483 ms/op 1.21
altair processBlock - 250000 vs - 7PWei worstcase 53.727 ms/op 52.940 ms/op 1.01
altair processBlock - 250000 vs - 7PWei worstcase hashState 71.602 ms/op 73.073 ms/op 0.98
phase0 processBlock - 250000 vs - 7PWei normalcase 2.9037 ms/op 2.0371 ms/op 1.43
phase0 processBlock - 250000 vs - 7PWei worstcase 32.137 ms/op 29.262 ms/op 1.10
altair processEth1Data - 250000 vs - 7PWei normalcase 513.40 us/op 481.73 us/op 1.07
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 7.4080 us/op 8.1860 us/op 0.90
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 24.200 us/op 24.926 us/op 0.97
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 8.9540 us/op 10.731 us/op 0.83
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 8.6590 us/op 8.3850 us/op 1.03
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 93.827 us/op 99.448 us/op 0.94
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 661.69 us/op 660.87 us/op 1.00
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 931.93 us/op 964.56 us/op 0.97
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 972.25 us/op 891.62 us/op 1.09
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.5433 ms/op 2.3562 ms/op 1.08
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.5431 ms/op 1.5385 ms/op 1.00
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.9331 ms/op 4.0644 ms/op 0.97
Tree 40 250000 create 302.45 ms/op 366.75 ms/op 0.82
Tree 40 250000 get(125000) 183.56 ns/op 192.72 ns/op 0.95
Tree 40 250000 set(125000) 945.65 ns/op 1.1741 us/op 0.81
Tree 40 250000 toArray() 20.799 ms/op 24.501 ms/op 0.85
Tree 40 250000 iterate all - toArray() + loop 18.097 ms/op 24.660 ms/op 0.73
Tree 40 250000 iterate all - get(i) 70.129 ms/op 79.340 ms/op 0.88
MutableVector 250000 create 10.786 ms/op 12.727 ms/op 0.85
MutableVector 250000 get(125000) 6.4830 ns/op 6.4110 ns/op 1.01
MutableVector 250000 set(125000) 266.85 ns/op 355.85 ns/op 0.75
MutableVector 250000 toArray() 2.9430 ms/op 4.0934 ms/op 0.72
MutableVector 250000 iterate all - toArray() + loop 3.3237 ms/op 4.4218 ms/op 0.75
MutableVector 250000 iterate all - get(i) 1.5345 ms/op 1.5540 ms/op 0.99
Array 250000 create 2.8351 ms/op 3.9646 ms/op 0.72
Array 250000 clone - spread 1.1276 ms/op 1.3101 ms/op 0.86
Array 250000 get(125000) 0.55700 ns/op 0.74600 ns/op 0.75
Array 250000 set(125000) 0.63700 ns/op 0.74400 ns/op 0.86
Array 250000 iterate all - loop 86.531 us/op 85.957 us/op 1.01
effectiveBalanceIncrements clone Uint8Array 300000 26.110 us/op 58.226 us/op 0.45
effectiveBalanceIncrements clone MutableVector 300000 339.00 ns/op 398.00 ns/op 0.85
effectiveBalanceIncrements rw all Uint8Array 300000 166.14 us/op 173.50 us/op 0.96
effectiveBalanceIncrements rw all MutableVector 300000 79.280 ms/op 110.78 ms/op 0.72
phase0 afterProcessEpoch - 250000 vs - 7PWei 117.46 ms/op 121.36 ms/op 0.97
phase0 beforeProcessEpoch - 250000 vs - 7PWei 46.092 ms/op 47.556 ms/op 0.97
altair processEpoch - mainnet_e81889 305.33 ms/op 366.76 ms/op 0.83
mainnet_e81889 - altair beforeProcessEpoch 50.336 ms/op 77.793 ms/op 0.65
mainnet_e81889 - altair processJustificationAndFinalization 19.489 us/op 22.756 us/op 0.86
mainnet_e81889 - altair processInactivityUpdates 6.1370 ms/op 8.8625 ms/op 0.69
mainnet_e81889 - altair processRewardsAndPenalties 69.156 ms/op 72.379 ms/op 0.96
mainnet_e81889 - altair processRegistryUpdates 2.8860 us/op 6.8040 us/op 0.42
mainnet_e81889 - altair processSlashings 567.00 ns/op 589.00 ns/op 0.96
mainnet_e81889 - altair processEth1DataReset 594.00 ns/op 665.00 ns/op 0.89
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2781 ms/op 1.2837 ms/op 1.00
mainnet_e81889 - altair processSlashingsReset 4.6160 us/op 6.8960 us/op 0.67
mainnet_e81889 - altair processRandaoMixesReset 6.7830 us/op 10.780 us/op 0.63
mainnet_e81889 - altair processHistoricalRootsUpdate 819.00 ns/op 1.4570 us/op 0.56
mainnet_e81889 - altair processParticipationFlagUpdates 2.6320 us/op 3.6040 us/op 0.73
mainnet_e81889 - altair processSyncCommitteeUpdates 642.00 ns/op 774.00 ns/op 0.83
mainnet_e81889 - altair afterProcessEpoch 123.39 ms/op 135.75 ms/op 0.91
phase0 processEpoch - mainnet_e58758 320.45 ms/op 409.57 ms/op 0.78
mainnet_e58758 - phase0 beforeProcessEpoch 118.01 ms/op 141.70 ms/op 0.83
mainnet_e58758 - phase0 processJustificationAndFinalization 15.964 us/op 23.611 us/op 0.68
mainnet_e58758 - phase0 processRewardsAndPenalties 41.270 ms/op 67.543 ms/op 0.61
mainnet_e58758 - phase0 processRegistryUpdates 7.9120 us/op 11.567 us/op 0.68
mainnet_e58758 - phase0 processSlashings 582.00 ns/op 676.00 ns/op 0.86
mainnet_e58758 - phase0 processEth1DataReset 599.00 ns/op 845.00 ns/op 0.71
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0009 ms/op 1.0583 ms/op 0.95
mainnet_e58758 - phase0 processSlashingsReset 2.9860 us/op 5.3830 us/op 0.55
mainnet_e58758 - phase0 processRandaoMixesReset 4.4580 us/op 7.9350 us/op 0.56
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.0830 us/op 1.2040 us/op 0.90
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.8130 us/op 7.0770 us/op 0.68
mainnet_e58758 - phase0 afterProcessEpoch 94.222 ms/op 102.10 ms/op 0.92
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2298 ms/op 1.2837 ms/op 0.96
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4855 ms/op 2.0535 ms/op 0.72
altair processInactivityUpdates - 250000 normalcase 25.148 ms/op 26.833 ms/op 0.94
altair processInactivityUpdates - 250000 worstcase 25.064 ms/op 26.122 ms/op 0.96
phase0 processRegistryUpdates - 250000 normalcase 7.5200 us/op 7.5790 us/op 0.99
phase0 processRegistryUpdates - 250000 badcase_full_deposits 235.33 us/op 317.62 us/op 0.74
phase0 processRegistryUpdates - 250000 worstcase 0.5 126.64 ms/op 139.26 ms/op 0.91
altair processRewardsAndPenalties - 250000 normalcase 68.073 ms/op 71.401 ms/op 0.95
altair processRewardsAndPenalties - 250000 worstcase 69.781 ms/op 70.609 ms/op 0.99
phase0 getAttestationDeltas - 250000 normalcase 6.9537 ms/op 8.6068 ms/op 0.81
phase0 getAttestationDeltas - 250000 worstcase 6.4170 ms/op 7.0669 ms/op 0.91
phase0 processSlashings - 250000 worstcase 3.3550 ms/op 3.3702 ms/op 1.00
altair processSyncCommitteeUpdates - 250000 168.38 ms/op 193.69 ms/op 0.87
BeaconState.hashTreeRoot - No change 249.00 ns/op 267.00 ns/op 0.93
BeaconState.hashTreeRoot - 1 full validator 53.494 us/op 53.914 us/op 0.99
BeaconState.hashTreeRoot - 32 full validator 561.55 us/op 537.22 us/op 1.05
BeaconState.hashTreeRoot - 512 full validator 4.8810 ms/op 5.1898 ms/op 0.94
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 61.188 us/op 67.376 us/op 0.91
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 863.10 us/op 904.29 us/op 0.95
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.631 ms/op 13.809 ms/op 0.84
BeaconState.hashTreeRoot - 1 balances 47.218 us/op 51.686 us/op 0.91
BeaconState.hashTreeRoot - 32 balances 457.43 us/op 492.56 us/op 0.93
BeaconState.hashTreeRoot - 512 balances 4.3717 ms/op 5.2247 ms/op 0.84
BeaconState.hashTreeRoot - 250000 balances 76.233 ms/op 77.997 ms/op 0.98
aggregationBits - 2048 els - zipIndexesInBitList 15.402 us/op 19.124 us/op 0.81
regular array get 100000 times 32.923 us/op 44.499 us/op 0.74
wrappedArray get 100000 times 33.194 us/op 33.851 us/op 0.98
arrayWithProxy get 100000 times 15.998 ms/op 16.048 ms/op 1.00
ssz.Root.equals 557.00 ns/op 575.00 ns/op 0.97
byteArrayEquals 568.00 ns/op 581.00 ns/op 0.98
shuffle list - 16384 els 7.0421 ms/op 7.5149 ms/op 0.94
shuffle list - 250000 els 103.07 ms/op 107.41 ms/op 0.96
processSlot - 1 slots 8.5970 us/op 9.5120 us/op 0.90
processSlot - 32 slots 1.3811 ms/op 1.5120 ms/op 0.91
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 37.877 ms/op 37.190 ms/op 1.02
getCommitteeAssignments - req 1 vs - 250000 vc 2.8012 ms/op 2.9358 ms/op 0.95
getCommitteeAssignments - req 100 vs - 250000 vc 3.9941 ms/op 4.2352 ms/op 0.94
getCommitteeAssignments - req 1000 vs - 250000 vc 4.3236 ms/op 4.5577 ms/op 0.95
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.6000 ns/op 4.8800 ns/op 0.94
state getBlockRootAtSlot - 250000 vs - 7PWei 934.86 ns/op 654.88 ns/op 1.43
computeProposers - vc 250000 10.673 ms/op 11.144 ms/op 0.96
computeEpochShuffling - vc 250000 102.59 ms/op 107.59 ms/op 0.95
getNextSyncCommittee - vc 250000 174.21 ms/op 183.17 ms/op 0.95
computeSigningRoot for AttestationData 13.712 us/op 14.524 us/op 0.94
hash AttestationData serialized data then Buffer.toString(base64) 2.4839 us/op 2.5466 us/op 0.98
toHexString serialized data 1.0728 us/op 1.1850 us/op 0.91
Buffer.toString(base64) 324.49 ns/op 366.08 ns/op 0.89

by benchmarkbot/action

@philknows philknows added this to the v1.10.0 milestone Jun 23, 2023
@wemeetagain wemeetagain merged commit 50daa01 into unstable Jun 23, 2023
@wemeetagain wemeetagain deleted the nflaig/availableParallelism branch June 23, 2023 18:58
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.10.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants