Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: pull block from peers after a cutoff if corresponding gossip blobs are seen #6534

Merged
merged 3 commits into from
Mar 16, 2024

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Mar 12, 2024

generally we now start block import as soon as we see a block and pull blobs if not all blobs seen.
However in edgecases, even block might arrive too late

This PR starts pulling the block (and remaining blobs) from peers if we don't see a block past a certain cutoff

@g11tech g11tech requested a review from a team as a code owner March 12, 2024 17:08
Copy link

codecov bot commented Mar 12, 2024

Codecov Report

Merging #6534 (79e2887) into unstable (9116526) will decrease coverage by 0.14%.
Report is 7 commits behind head on unstable.
The diff coverage is 22.92%.

Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #6534      +/-   ##
============================================
- Coverage     61.53%   61.39%   -0.14%     
============================================
  Files           556      556              
  Lines         58647    58751     +104     
  Branches       1847     1848       +1     
============================================
- Hits          36087    36073      -14     
- Misses        22520    22638     +118     
  Partials         40       40              

Copy link
Contributor

github-actions bot commented Mar 12, 2024

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: 9103feb Previous: 30d347d Ratio
Object set x1000 24.947 ns/op 84.033 ns/op 0.30
Map set x1000 18.807 ns/op 66.266 ns/op 0.28
Full benchmark results
Benchmark suite Current: 9103feb Previous: 30d347d Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 342.17 us/op 613.34 us/op 0.56
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 50.602 us/op 135.55 us/op 0.37
BLS verify - blst-native 1.1127 ms/op 1.4424 ms/op 0.77
BLS verifyMultipleSignatures 3 - blst-native 2.3882 ms/op 3.1719 ms/op 0.75
BLS verifyMultipleSignatures 8 - blst-native 5.2288 ms/op 6.3607 ms/op 0.82
BLS verifyMultipleSignatures 32 - blst-native 19.173 ms/op 23.896 ms/op 0.80
BLS verifyMultipleSignatures 64 - blst-native 37.963 ms/op 48.418 ms/op 0.78
BLS verifyMultipleSignatures 128 - blst-native 76.594 ms/op 104.92 ms/op 0.73
BLS deserializing 10000 signatures 797.31 ms/op 1.0126 s/op 0.79
BLS deserializing 100000 signatures 8.4169 s/op 9.6155 s/op 0.88
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.1862 ms/op 1.3965 ms/op 0.85
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.2848 ms/op 1.6346 ms/op 0.79
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.0499 ms/op 2.4605 ms/op 0.83
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.8737 ms/op 3.6875 ms/op 1.05
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.1767 ms/op 5.9505 ms/op 0.87
BLS aggregatePubkeys 32 - blst-native 22.623 us/op 27.117 us/op 0.83
BLS aggregatePubkeys 128 - blst-native 88.472 us/op 105.74 us/op 0.84
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 67.217 ms/op 79.504 ms/op 0.85
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 60.294 ms/op 65.196 ms/op 0.92
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 31.485 ms/op 53.384 ms/op 0.59
getSlashingsAndExits - default max 227.44 us/op 342.71 us/op 0.66
getSlashingsAndExits - 2k 310.35 us/op 647.92 us/op 0.48
proposeBlockBody type=full, size=empty 4.1182 ms/op 6.7986 ms/op 0.61
isKnown best case - 1 super set check 345.00 ns/op 634.00 ns/op 0.54
isKnown normal case - 2 super set checks 336.00 ns/op 625.00 ns/op 0.54
isKnown worse case - 16 super set checks 334.00 ns/op 748.00 ns/op 0.45
CheckpointStateCache - add get delete 5.7940 us/op 8.8930 us/op 0.65
validate api signedAggregateAndProof - struct 2.4178 ms/op 3.0011 ms/op 0.81
validate gossip signedAggregateAndProof - struct 2.4251 ms/op 2.9346 ms/op 0.83
validate gossip attestation - vc 640000 1.1434 ms/op 1.4825 ms/op 0.77
batch validate gossip attestation - vc 640000 - chunk 32 139.23 us/op 181.28 us/op 0.77
batch validate gossip attestation - vc 640000 - chunk 64 125.18 us/op 155.18 us/op 0.81
batch validate gossip attestation - vc 640000 - chunk 128 116.27 us/op 156.22 us/op 0.74
batch validate gossip attestation - vc 640000 - chunk 256 120.55 us/op 162.55 us/op 0.74
pickEth1Vote - no votes 922.20 us/op 1.4471 ms/op 0.64
pickEth1Vote - max votes 9.4244 ms/op 12.410 ms/op 0.76
pickEth1Vote - Eth1Data hashTreeRoot value x2048 14.020 ms/op 24.653 ms/op 0.57
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 28.512 ms/op 35.233 ms/op 0.81
pickEth1Vote - Eth1Data fastSerialize value x2048 437.53 us/op 953.66 us/op 0.46
pickEth1Vote - Eth1Data fastSerialize tree x2048 8.9667 ms/op 6.0002 ms/op 1.49
bytes32 toHexString 476.00 ns/op 929.00 ns/op 0.51
bytes32 Buffer.toString(hex) 329.00 ns/op 339.00 ns/op 0.97
bytes32 Buffer.toString(hex) from Uint8Array 439.00 ns/op 579.00 ns/op 0.76
bytes32 Buffer.toString(hex) + 0x 332.00 ns/op 353.00 ns/op 0.94
Object access 1 prop 0.22300 ns/op 0.24000 ns/op 0.93
Map access 1 prop 0.20400 ns/op 0.18400 ns/op 1.11
Object get x1000 5.5310 ns/op 8.3970 ns/op 0.66
Map get x1000 0.83300 ns/op 0.99800 ns/op 0.83
Object set x1000 24.947 ns/op 84.033 ns/op 0.30
Map set x1000 18.807 ns/op 66.266 ns/op 0.28
Return object 10000 times 0.23880 ns/op 0.28890 ns/op 0.83
Throw Error 10000 times 2.7457 us/op 4.5554 us/op 0.60
fastMsgIdFn sha256 / 200 bytes 2.0530 us/op 4.0910 us/op 0.50
fastMsgIdFn h32 xxhash / 200 bytes 352.00 ns/op 427.00 ns/op 0.82
fastMsgIdFn h64 xxhash / 200 bytes 373.00 ns/op 456.00 ns/op 0.82
fastMsgIdFn sha256 / 1000 bytes 6.4990 us/op 13.102 us/op 0.50
fastMsgIdFn h32 xxhash / 1000 bytes 478.00 ns/op 543.00 ns/op 0.88
fastMsgIdFn h64 xxhash / 1000 bytes 462.00 ns/op 598.00 ns/op 0.77
fastMsgIdFn sha256 / 10000 bytes 61.483 us/op 120.35 us/op 0.51
fastMsgIdFn h32 xxhash / 10000 bytes 2.2440 us/op 2.5050 us/op 0.90
fastMsgIdFn h64 xxhash / 10000 bytes 1.5800 us/op 1.6780 us/op 0.94
send data - 1000 256B messages 16.228 ms/op 25.991 ms/op 0.62
send data - 1000 512B messages 22.114 ms/op 33.049 ms/op 0.67
send data - 1000 1024B messages 36.805 ms/op 53.948 ms/op 0.68
send data - 1000 1200B messages 46.262 ms/op 61.253 ms/op 0.76
send data - 1000 2048B messages 69.612 ms/op 62.167 ms/op 1.12
send data - 1000 4096B messages 41.296 ms/op 49.412 ms/op 0.84
send data - 1000 16384B messages 94.858 ms/op 121.59 ms/op 0.78
send data - 1000 65536B messages 430.12 ms/op 520.54 ms/op 0.83
enrSubnets - fastDeserialize 64 bits 1.1450 us/op 1.2280 us/op 0.93
enrSubnets - ssz BitVector 64 bits 466.00 ns/op 417.00 ns/op 1.12
enrSubnets - fastDeserialize 4 bits 210.00 ns/op 173.00 ns/op 1.21
enrSubnets - ssz BitVector 4 bits 430.00 ns/op 419.00 ns/op 1.03
prioritizePeers score -10:0 att 32-0.1 sync 2-0 65.474 us/op 98.789 us/op 0.66
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 78.636 us/op 134.98 us/op 0.58
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 104.54 us/op 173.45 us/op 0.60
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 178.11 us/op 295.58 us/op 0.60
prioritizePeers score 0:0 att 64-1 sync 4-1 200.28 us/op 356.23 us/op 0.56
array of 16000 items push then shift 1.2792 us/op 1.6319 us/op 0.78
LinkedList of 16000 items push then shift 6.1060 ns/op 8.9200 ns/op 0.68
array of 16000 items push then pop 92.328 ns/op 90.828 ns/op 1.02
LinkedList of 16000 items push then pop 5.9450 ns/op 9.2550 ns/op 0.64
array of 24000 items push then shift 1.8758 us/op 2.4847 us/op 0.75
LinkedList of 24000 items push then shift 6.7910 ns/op 9.1470 ns/op 0.74
array of 24000 items push then pop 94.121 ns/op 129.00 ns/op 0.73
LinkedList of 24000 items push then pop 5.8410 ns/op 8.7920 ns/op 0.66
intersect bitArray bitLen 8 4.8410 ns/op 5.8950 ns/op 0.82
intersect array and set length 8 54.922 ns/op 64.757 ns/op 0.85
intersect bitArray bitLen 128 29.873 ns/op 35.996 ns/op 0.83
intersect array and set length 128 760.88 ns/op 863.34 ns/op 0.88
bitArray.getTrueBitIndexes() bitLen 128 1.2780 us/op 1.4130 us/op 0.90
bitArray.getTrueBitIndexes() bitLen 248 1.9370 us/op 2.5060 us/op 0.77
bitArray.getTrueBitIndexes() bitLen 512 3.5040 us/op 4.7650 us/op 0.74
Buffer.concat 32 items 868.00 ns/op 992.00 ns/op 0.88
Uint8Array.set 32 items 1.7000 us/op 1.7360 us/op 0.98
Set add up to 64 items then delete first 1.7588 us/op 4.4206 us/op 0.40
OrderedSet add up to 64 items then delete first 2.6874 us/op 5.6065 us/op 0.48
Set add up to 64 items then delete last 2.0019 us/op 4.9519 us/op 0.40
OrderedSet add up to 64 items then delete last 2.9681 us/op 6.4759 us/op 0.46
Set add up to 64 items then delete middle 2.0061 us/op 5.4119 us/op 0.37
OrderedSet add up to 64 items then delete middle 4.2378 us/op 8.4436 us/op 0.50
Set add up to 128 items then delete first 3.8811 us/op 10.974 us/op 0.35
OrderedSet add up to 128 items then delete first 6.2542 us/op 13.468 us/op 0.46
Set add up to 128 items then delete last 3.8036 us/op 10.613 us/op 0.36
OrderedSet add up to 128 items then delete last 5.7708 us/op 15.037 us/op 0.38
Set add up to 128 items then delete middle 3.9859 us/op 10.414 us/op 0.38
OrderedSet add up to 128 items then delete middle 10.543 us/op 18.050 us/op 0.58
Set add up to 256 items then delete first 7.5026 us/op 19.689 us/op 0.38
OrderedSet add up to 256 items then delete first 11.912 us/op 24.834 us/op 0.48
Set add up to 256 items then delete last 7.4603 us/op 19.965 us/op 0.37
OrderedSet add up to 256 items then delete last 11.230 us/op 24.946 us/op 0.45
Set add up to 256 items then delete middle 7.4335 us/op 20.911 us/op 0.36
OrderedSet add up to 256 items then delete middle 31.229 us/op 48.036 us/op 0.65
transfer serialized Status (84 B) 1.2870 us/op 1.8120 us/op 0.71
copy serialized Status (84 B) 1.0080 us/op 1.3780 us/op 0.73
transfer serialized SignedVoluntaryExit (112 B) 1.3160 us/op 1.9420 us/op 0.68
copy serialized SignedVoluntaryExit (112 B) 1.0320 us/op 1.5460 us/op 0.67
transfer serialized ProposerSlashing (416 B) 1.4380 us/op 2.4210 us/op 0.59
copy serialized ProposerSlashing (416 B) 1.7730 us/op 2.0530 us/op 0.86
transfer serialized Attestation (485 B) 2.4490 us/op 3.2260 us/op 0.76
copy serialized Attestation (485 B) 2.3650 us/op 2.2870 us/op 1.03
transfer serialized AttesterSlashing (33232 B) 2.3930 us/op 2.6490 us/op 0.90
copy serialized AttesterSlashing (33232 B) 3.8470 us/op 7.1250 us/op 0.54
transfer serialized Small SignedBeaconBlock (128000 B) 2.5330 us/op 3.4820 us/op 0.73
copy serialized Small SignedBeaconBlock (128000 B) 12.613 us/op 19.771 us/op 0.64
transfer serialized Avg SignedBeaconBlock (200000 B) 2.6200 us/op 3.6770 us/op 0.71
copy serialized Avg SignedBeaconBlock (200000 B) 18.400 us/op 30.664 us/op 0.60
transfer serialized BlobsSidecar (524380 B) 2.3510 us/op 3.6960 us/op 0.64
copy serialized BlobsSidecar (524380 B) 117.45 us/op 96.083 us/op 1.22
transfer serialized Big SignedBeaconBlock (1000000 B) 2.3650 us/op 3.5370 us/op 0.67
copy serialized Big SignedBeaconBlock (1000000 B) 128.17 us/op 181.27 us/op 0.71
pass gossip attestations to forkchoice per slot 2.7426 ms/op 4.7464 ms/op 0.58
forkChoice updateHead vc 100000 bc 64 eq 0 451.05 us/op 754.80 us/op 0.60
forkChoice updateHead vc 600000 bc 64 eq 0 3.0222 ms/op 6.8560 ms/op 0.44
forkChoice updateHead vc 1000000 bc 64 eq 0 4.3753 ms/op 8.3082 ms/op 0.53
forkChoice updateHead vc 600000 bc 320 eq 0 2.6235 ms/op 4.6675 ms/op 0.56
forkChoice updateHead vc 600000 bc 1200 eq 0 2.6449 ms/op 4.8891 ms/op 0.54
forkChoice updateHead vc 600000 bc 7200 eq 0 3.1803 ms/op 7.0563 ms/op 0.45
forkChoice updateHead vc 600000 bc 64 eq 1000 9.4257 ms/op 12.766 ms/op 0.74
forkChoice updateHead vc 600000 bc 64 eq 10000 9.4648 ms/op 13.263 ms/op 0.71
forkChoice updateHead vc 600000 bc 64 eq 300000 11.999 ms/op 21.063 ms/op 0.57
computeDeltas 500000 validators 300 proto nodes 3.1585 ms/op 7.5273 ms/op 0.42
computeDeltas 500000 validators 1200 proto nodes 2.9916 ms/op 7.4067 ms/op 0.40
computeDeltas 500000 validators 7200 proto nodes 3.8212 ms/op 6.8944 ms/op 0.55
computeDeltas 750000 validators 300 proto nodes 5.4336 ms/op 10.672 ms/op 0.51
computeDeltas 750000 validators 1200 proto nodes 4.6154 ms/op 10.378 ms/op 0.44
computeDeltas 750000 validators 7200 proto nodes 5.0028 ms/op 10.189 ms/op 0.49
computeDeltas 1400000 validators 300 proto nodes 8.8190 ms/op 21.615 ms/op 0.41
computeDeltas 1400000 validators 1200 proto nodes 8.6691 ms/op 21.533 ms/op 0.40
computeDeltas 1400000 validators 7200 proto nodes 8.9793 ms/op 22.253 ms/op 0.40
computeDeltas 2100000 validators 300 proto nodes 13.936 ms/op 32.441 ms/op 0.43
computeDeltas 2100000 validators 1200 proto nodes 13.091 ms/op 31.957 ms/op 0.41
computeDeltas 2100000 validators 7200 proto nodes 13.392 ms/op 34.114 ms/op 0.39
altair processAttestation - 250000 vs - 7PWei normalcase 1.5367 ms/op 3.4623 ms/op 0.44
altair processAttestation - 250000 vs - 7PWei worstcase 2.7843 ms/op 5.0574 ms/op 0.55
altair processAttestation - setStatus - 1/6 committees join 81.236 us/op 214.05 us/op 0.38
altair processAttestation - setStatus - 1/3 committees join 148.31 us/op 438.67 us/op 0.34
altair processAttestation - setStatus - 1/2 committees join 223.18 us/op 633.90 us/op 0.35
altair processAttestation - setStatus - 2/3 committees join 280.63 us/op 707.69 us/op 0.40
altair processAttestation - setStatus - 4/5 committees join 402.20 us/op 1.0624 ms/op 0.38
altair processAttestation - setStatus - 100% committees join 474.57 us/op 1.1025 ms/op 0.43
altair processBlock - 250000 vs - 7PWei normalcase 7.6590 ms/op 12.780 ms/op 0.60
altair processBlock - 250000 vs - 7PWei normalcase hashState 30.113 ms/op 42.296 ms/op 0.71
altair processBlock - 250000 vs - 7PWei worstcase 32.335 ms/op 51.156 ms/op 0.63
altair processBlock - 250000 vs - 7PWei worstcase hashState 84.132 ms/op 125.16 ms/op 0.67
phase0 processBlock - 250000 vs - 7PWei normalcase 2.6027 ms/op 4.4179 ms/op 0.59
phase0 processBlock - 250000 vs - 7PWei worstcase 26.561 ms/op 41.948 ms/op 0.63
altair processEth1Data - 250000 vs - 7PWei normalcase 424.63 us/op 789.68 us/op 0.54
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 8.6110 us/op 21.420 us/op 0.40
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 40.235 us/op 70.533 us/op 0.57
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 18.303 us/op 30.090 us/op 0.61
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 12.704 us/op 24.640 us/op 0.52
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 134.76 us/op 201.81 us/op 0.67
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.4754 ms/op 1.6953 ms/op 0.87
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.3505 ms/op 2.7625 ms/op 0.49
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.1615 ms/op 2.2803 ms/op 0.51
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.2897 ms/op 5.5456 ms/op 0.41
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.9541 ms/op 3.0996 ms/op 0.63
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 4.0968 ms/op 7.0466 ms/op 0.58
Tree 40 250000 create 313.15 ms/op 598.16 ms/op 0.52
Tree 40 250000 get(125000) 116.62 ns/op 230.75 ns/op 0.51
Tree 40 250000 set(125000) 828.33 ns/op 1.5979 us/op 0.52
Tree 40 250000 toArray() 18.302 ms/op 26.490 ms/op 0.69
Tree 40 250000 iterate all - toArray() + loop 17.839 ms/op 29.353 ms/op 0.61
Tree 40 250000 iterate all - get(i) 48.229 ms/op 83.643 ms/op 0.58
MutableVector 250000 create 13.209 ms/op 17.584 ms/op 0.75
MutableVector 250000 get(125000) 5.8220 ns/op 7.1030 ns/op 0.82
MutableVector 250000 set(125000) 205.66 ns/op 547.36 ns/op 0.38
MutableVector 250000 toArray() 2.5000 ms/op 4.3912 ms/op 0.57
MutableVector 250000 iterate all - toArray() + loop 3.4938 ms/op 4.7697 ms/op 0.73
MutableVector 250000 iterate all - get(i) 1.3195 ms/op 1.6118 ms/op 0.82
Array 250000 create 2.9274 ms/op 3.9774 ms/op 0.74
Array 250000 clone - spread 1.1989 ms/op 1.4693 ms/op 0.82
Array 250000 get(125000) 1.0660 ns/op 1.2850 ns/op 0.83
Array 250000 set(125000) 1.3240 ns/op 4.9820 ns/op 0.27
Array 250000 iterate all - loop 150.45 us/op 180.43 us/op 0.83
effectiveBalanceIncrements clone Uint8Array 300000 28.240 us/op 51.058 us/op 0.55
effectiveBalanceIncrements clone MutableVector 300000 435.00 ns/op 468.00 ns/op 0.93
effectiveBalanceIncrements rw all Uint8Array 300000 183.63 us/op 210.10 us/op 0.87
effectiveBalanceIncrements rw all MutableVector 300000 73.671 ms/op 101.05 ms/op 0.73
phase0 afterProcessEpoch - 250000 vs - 7PWei 74.415 ms/op 124.54 ms/op 0.60
phase0 beforeProcessEpoch - 250000 vs - 7PWei 34.130 ms/op 68.793 ms/op 0.50
altair processEpoch - mainnet_e81889 395.48 ms/op 498.69 ms/op 0.79
mainnet_e81889 - altair beforeProcessEpoch 69.783 ms/op 93.002 ms/op 0.75
mainnet_e81889 - altair processJustificationAndFinalization 11.470 us/op 20.016 us/op 0.57
mainnet_e81889 - altair processInactivityUpdates 5.1484 ms/op 8.1375 ms/op 0.63
mainnet_e81889 - altair processRewardsAndPenalties 61.088 ms/op 67.131 ms/op 0.91
mainnet_e81889 - altair processRegistryUpdates 1.6990 us/op 6.6160 us/op 0.26
mainnet_e81889 - altair processSlashings 518.00 ns/op 920.00 ns/op 0.56
mainnet_e81889 - altair processEth1DataReset 609.00 ns/op 1.1240 us/op 0.54
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.0845 ms/op 1.8321 ms/op 0.59
mainnet_e81889 - altair processSlashingsReset 2.2270 us/op 8.6520 us/op 0.26
mainnet_e81889 - altair processRandaoMixesReset 3.2170 us/op 7.9730 us/op 0.40
mainnet_e81889 - altair processHistoricalRootsUpdate 834.00 ns/op 1.1150 us/op 0.75
mainnet_e81889 - altair processParticipationFlagUpdates 2.8900 us/op 4.8380 us/op 0.60
mainnet_e81889 - altair processSyncCommitteeUpdates 696.00 ns/op 1.4610 us/op 0.48
mainnet_e81889 - altair afterProcessEpoch 80.401 ms/op 130.21 ms/op 0.62
capella processEpoch - mainnet_e217614 1.6315 s/op 2.1085 s/op 0.77
mainnet_e217614 - capella beforeProcessEpoch 434.08 ms/op 595.45 ms/op 0.73
mainnet_e217614 - capella processJustificationAndFinalization 16.854 us/op 23.846 us/op 0.71
mainnet_e217614 - capella processInactivityUpdates 17.787 ms/op 32.287 ms/op 0.55
mainnet_e217614 - capella processRewardsAndPenalties 313.91 ms/op 433.80 ms/op 0.72
mainnet_e217614 - capella processRegistryUpdates 13.157 us/op 37.533 us/op 0.35
mainnet_e217614 - capella processSlashings 664.00 ns/op 1.0630 us/op 0.62
mainnet_e217614 - capella processEth1DataReset 508.00 ns/op 1.0240 us/op 0.50
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.0849 ms/op 10.447 ms/op 0.39
mainnet_e217614 - capella processSlashingsReset 1.6010 us/op 4.1340 us/op 0.39
mainnet_e217614 - capella processRandaoMixesReset 3.0760 us/op 8.9350 us/op 0.34
mainnet_e217614 - capella processHistoricalRootsUpdate 631.00 ns/op 1.0490 us/op 0.60
mainnet_e217614 - capella processParticipationFlagUpdates 1.3590 us/op 2.4780 us/op 0.55
mainnet_e217614 - capella afterProcessEpoch 194.70 ms/op 329.74 ms/op 0.59
phase0 processEpoch - mainnet_e58758 408.70 ms/op 599.64 ms/op 0.68
mainnet_e58758 - phase0 beforeProcessEpoch 118.42 ms/op 190.45 ms/op 0.62
mainnet_e58758 - phase0 processJustificationAndFinalization 15.043 us/op 31.475 us/op 0.48
mainnet_e58758 - phase0 processRewardsAndPenalties 41.324 ms/op 67.400 ms/op 0.61
mainnet_e58758 - phase0 processRegistryUpdates 8.1330 us/op 18.779 us/op 0.43
mainnet_e58758 - phase0 processSlashings 596.00 ns/op 1.1000 us/op 0.54
mainnet_e58758 - phase0 processEth1DataReset 534.00 ns/op 989.00 ns/op 0.54
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.3989 ms/op 1.4040 ms/op 1.00
mainnet_e58758 - phase0 processSlashingsReset 2.1700 us/op 5.7510 us/op 0.38
mainnet_e58758 - phase0 processRandaoMixesReset 2.4640 us/op 7.0910 us/op 0.35
mainnet_e58758 - phase0 processHistoricalRootsUpdate 584.00 ns/op 801.00 ns/op 0.73
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.4890 us/op 9.4400 us/op 0.37
mainnet_e58758 - phase0 afterProcessEpoch 66.830 ms/op 110.67 ms/op 0.60
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.0198 ms/op 1.6310 ms/op 0.63
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.0669 ms/op 1.8752 ms/op 0.57
altair processInactivityUpdates - 250000 normalcase 24.785 ms/op 41.438 ms/op 0.60
altair processInactivityUpdates - 250000 worstcase 23.013 ms/op 39.676 ms/op 0.58
phase0 processRegistryUpdates - 250000 normalcase 6.3370 us/op 12.499 us/op 0.51
phase0 processRegistryUpdates - 250000 badcase_full_deposits 410.53 us/op 428.95 us/op 0.96
phase0 processRegistryUpdates - 250000 worstcase 0.5 131.42 ms/op 150.81 ms/op 0.87
altair processRewardsAndPenalties - 250000 normalcase 58.568 ms/op 67.928 ms/op 0.86
altair processRewardsAndPenalties - 250000 worstcase 56.099 ms/op 63.856 ms/op 0.88
phase0 getAttestationDeltas - 250000 normalcase 5.5952 ms/op 12.111 ms/op 0.46
phase0 getAttestationDeltas - 250000 worstcase 7.1827 ms/op 11.825 ms/op 0.61
phase0 processSlashings - 250000 worstcase 83.408 us/op 112.72 us/op 0.74
altair processSyncCommitteeUpdates - 250000 104.34 ms/op 182.77 ms/op 0.57
BeaconState.hashTreeRoot - No change 379.00 ns/op 901.00 ns/op 0.42
BeaconState.hashTreeRoot - 1 full validator 153.42 us/op 178.00 us/op 0.86
BeaconState.hashTreeRoot - 32 full validator 1.6071 ms/op 1.7865 ms/op 0.90
BeaconState.hashTreeRoot - 512 full validator 15.071 ms/op 20.213 ms/op 0.75
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 154.26 us/op 195.21 us/op 0.79
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.8587 ms/op 2.5414 ms/op 0.73
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 28.143 ms/op 31.721 ms/op 0.89
BeaconState.hashTreeRoot - 1 balances 104.99 us/op 145.49 us/op 0.72
BeaconState.hashTreeRoot - 32 balances 1.3584 ms/op 1.2485 ms/op 1.09
BeaconState.hashTreeRoot - 512 balances 8.8058 ms/op 13.550 ms/op 0.65
BeaconState.hashTreeRoot - 250000 balances 200.94 ms/op 223.96 ms/op 0.90
aggregationBits - 2048 els - zipIndexesInBitList 20.056 us/op 34.626 us/op 0.58
byteArrayEquals 32 60.433 ns/op 81.127 ns/op 0.74
Buffer.compare 32 33.963 ns/op 60.817 ns/op 0.56
byteArrayEquals 1024 1.6230 us/op 2.1930 us/op 0.74
Buffer.compare 1024 40.988 ns/op 75.575 ns/op 0.54
byteArrayEquals 16384 27.506 us/op 34.530 us/op 0.80
Buffer.compare 16384 230.47 ns/op 283.78 ns/op 0.81
byteArrayEquals 123687377 208.20 ms/op 267.32 ms/op 0.78
Buffer.compare 123687377 4.3555 ms/op 9.5739 ms/op 0.45
byteArrayEquals 32 - diff last byte 61.433 ns/op 93.648 ns/op 0.66
Buffer.compare 32 - diff last byte 38.085 ns/op 64.079 ns/op 0.59
byteArrayEquals 1024 - diff last byte 1.7201 us/op 2.2746 us/op 0.76
Buffer.compare 1024 - diff last byte 44.896 ns/op 82.480 ns/op 0.54
byteArrayEquals 16384 - diff last byte 27.599 us/op 36.225 us/op 0.76
Buffer.compare 16384 - diff last byte 234.55 ns/op 302.64 ns/op 0.78
byteArrayEquals 123687377 - diff last byte 209.23 ms/op 302.20 ms/op 0.69
Buffer.compare 123687377 - diff last byte 4.1971 ms/op 9.3600 ms/op 0.45
byteArrayEquals 32 - random bytes 4.6040 ns/op 7.0160 ns/op 0.66
Buffer.compare 32 - random bytes 39.735 ns/op 67.647 ns/op 0.59
byteArrayEquals 1024 - random bytes 4.4790 ns/op 7.0150 ns/op 0.64
Buffer.compare 1024 - random bytes 37.674 ns/op 75.458 ns/op 0.50
byteArrayEquals 16384 - random bytes 4.4240 ns/op 6.9890 ns/op 0.63
Buffer.compare 16384 - random bytes 37.694 ns/op 74.911 ns/op 0.50
byteArrayEquals 123687377 - random bytes 7.9200 ns/op 16.540 ns/op 0.48
Buffer.compare 123687377 - random bytes 44.650 ns/op 86.900 ns/op 0.51
regular array get 100000 times 40.891 us/op 51.543 us/op 0.79
wrappedArray get 100000 times 40.870 us/op 57.112 us/op 0.72
arrayWithProxy get 100000 times 10.908 ms/op 15.807 ms/op 0.69
ssz.Root.equals 54.174 ns/op 62.478 ns/op 0.87
byteArrayEquals 51.795 ns/op 61.535 ns/op 0.84
Buffer.compare 8.5710 ns/op 13.573 ns/op 0.63
shuffle list - 16384 els 5.7666 ms/op 9.7446 ms/op 0.59
shuffle list - 250000 els 87.321 ms/op 139.82 ms/op 0.62
processSlot - 1 slots 17.560 us/op 23.728 us/op 0.74
processSlot - 32 slots 4.0470 ms/op 3.8091 ms/op 1.06
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 48.664 ms/op 61.862 ms/op 0.79
getCommitteeAssignments - req 1 vs - 250000 vc 2.4058 ms/op 2.8629 ms/op 0.84
getCommitteeAssignments - req 100 vs - 250000 vc 3.6392 ms/op 4.2981 ms/op 0.85
getCommitteeAssignments - req 1000 vs - 250000 vc 3.9276 ms/op 4.5547 ms/op 0.86
findModifiedValidators - 10000 modified validators 403.74 ms/op 634.80 ms/op 0.64
findModifiedValidators - 1000 modified validators 342.03 ms/op 465.23 ms/op 0.74
findModifiedValidators - 100 modified validators 309.30 ms/op 440.70 ms/op 0.70
findModifiedValidators - 10 modified validators 348.90 ms/op 456.84 ms/op 0.76
findModifiedValidators - 1 modified validators 313.19 ms/op 441.75 ms/op 0.71
findModifiedValidators - no difference 319.27 ms/op 450.12 ms/op 0.71
compare ViewDUs 4.1081 s/op 6.3048 s/op 0.65
compare each validator Uint8Array 1.5616 s/op 1.8497 s/op 0.84
compare ViewDU to Uint8Array 807.29 ms/op 1.5313 s/op 0.53
migrate state 1000000 validators, 24 modified, 0 new 662.30 ms/op 835.10 ms/op 0.79
migrate state 1000000 validators, 1700 modified, 1000 new 934.84 ms/op 1.1160 s/op 0.84
migrate state 1000000 validators, 3400 modified, 2000 new 1.1804 s/op 1.3660 s/op 0.86
migrate state 1500000 validators, 24 modified, 0 new 691.92 ms/op 848.13 ms/op 0.82
migrate state 1500000 validators, 1700 modified, 1000 new 894.89 ms/op 1.2125 s/op 0.74
migrate state 1500000 validators, 3400 modified, 2000 new 1.0841 s/op 1.4029 s/op 0.77
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.3200 ns/op 4.7400 ns/op 0.91
state getBlockRootAtSlot - 250000 vs - 7PWei 534.03 ns/op 993.60 ns/op 0.54
computeProposers - vc 250000 7.7898 ms/op 10.159 ms/op 0.77
computeEpochShuffling - vc 250000 88.913 ms/op 129.98 ms/op 0.68
getNextSyncCommittee - vc 250000 120.38 ms/op 176.26 ms/op 0.68
computeSigningRoot for AttestationData 27.161 us/op 25.973 us/op 1.05
hash AttestationData serialized data then Buffer.toString(base64) 1.2972 us/op 2.3719 us/op 0.55
toHexString serialized data 848.78 ns/op 1.1286 us/op 0.75
Buffer.toString(base64) 166.63 ns/op 259.31 ns/op 0.64

by benchmarkbot/action

resolveAvailability
);

resolveBlock(blockInput);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in this case there are not enough blobs showing up, why do we resolveBlock() here? if it's needed in both if and else then let's put in common place

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so that the waiting handler for blob can resolve and emit event to pull blobs (and add its peer over there)

For code cleanliness its ok to be replicated as the return follows this statement

// however we can emit the event which will atleast add the peer to the list of peers to pull
// data from
if (normalBlockInput.type === BlockInputType.blobsPromise) {
events.emit(NetworkEvent.unknownBlockInput, {blockInput: normalBlockInput, peer: peerIdStr});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after block comes and there are not enough blobs, we already emit this event in handleValidBeaconBlock(), then we also emit it here. Is this a duplicate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this is a duplicate aimed to add this peer to the peers for unknown blobs pull

// resolve the block if thats unavailable
let block, blobsCache, blockBytes, resolveAvailability;
if (unavailableBlockInput.block === null) {
const allBlocks = await network.sendBeaconBlocksByRoot(peerId, [fromHexString(unavailableBlockInput.blockRootHex)]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now the block source lodestar_import_block_by_source_total is very tricky, it could be wrong for now

for block, we want to know if it comes from gossip, or this function, or the above beaconBlocksMaybeBlobsByRoot() function

same for blobs, we want to know % of them coming from 3 different places as above

if it's too much for this PR please have a follow up PR or track in another issue @g11tech

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes lets do a followup on this

Copy link
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@g11tech g11tech merged commit 57127cc into unstable Mar 16, 2024
19 of 20 checks passed
@g11tech g11tech deleted the nullblockinput branch March 16, 2024 12:00
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.18.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants