Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: prometheus-gc-stats metrics bugfix #5683

Merged
merged 1 commit into from
Jun 22, 2023

Conversation

wemeetagain
Copy link
Member

@wemeetagain wemeetagain commented Jun 22, 2023

Motivation

The new prometheus-gc-stats was reporting gc time in microseconds instead of seconds, which our dashboards need.

Description

Update prometheus-gc-stats to the latest version which now reports in seconds

@wemeetagain wemeetagain requested a review from a team as a code owner June 22, 2023 15:07
@wemeetagain wemeetagain force-pushed the cayman/update-prometheus-gc-stats branch from e3c9112 to 42fa4dd Compare June 22, 2023 15:10
@wemeetagain wemeetagain changed the title Cayman/update prometheus gc stats fix: prometheus-gc-stats metrics bugfix Jun 22, 2023
@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 19327d0 Previous: 7c101f0 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 805.88 us/op 559.19 us/op 1.44
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 68.423 us/op 48.202 us/op 1.42
BLS verify - blst-native 1.6141 ms/op 1.2423 ms/op 1.30
BLS verifyMultipleSignatures 3 - blst-native 3.3037 ms/op 2.5463 ms/op 1.30
BLS verifyMultipleSignatures 8 - blst-native 7.0938 ms/op 5.4504 ms/op 1.30
BLS verifyMultipleSignatures 32 - blst-native 25.735 ms/op 19.677 ms/op 1.31
BLS aggregatePubkeys 32 - blst-native 34.052 us/op 26.415 us/op 1.29
BLS aggregatePubkeys 128 - blst-native 132.52 us/op 101.73 us/op 1.30
getAttestationsForBlock 84.072 ms/op 62.978 ms/op 1.33
isKnown best case - 1 super set check 403.00 ns/op 256.00 ns/op 1.57
isKnown normal case - 2 super set checks 394.00 ns/op 253.00 ns/op 1.56
isKnown worse case - 16 super set checks 391.00 ns/op 263.00 ns/op 1.49
CheckpointStateCache - add get delete 8.0030 us/op 5.4440 us/op 1.47
validate gossip signedAggregateAndProof - struct 3.7225 ms/op 2.7904 ms/op 1.33
validate gossip attestation - struct 1.7589 ms/op 1.3339 ms/op 1.32
pickEth1Vote - no votes 2.2136 ms/op 1.3203 ms/op 1.68
pickEth1Vote - max votes 15.613 ms/op 11.307 ms/op 1.38
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.929 ms/op 8.6479 ms/op 1.50
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.865 ms/op 14.647 ms/op 1.49
pickEth1Vote - Eth1Data fastSerialize value x2048 1.2740 ms/op 694.07 us/op 1.84
pickEth1Vote - Eth1Data fastSerialize tree x2048 12.902 ms/op 5.7362 ms/op 2.25
bytes32 toHexString 978.00 ns/op 651.00 ns/op 1.50
bytes32 Buffer.toString(hex) 670.00 ns/op 401.00 ns/op 1.67
bytes32 Buffer.toString(hex) from Uint8Array 1.0190 us/op 589.00 ns/op 1.73
bytes32 Buffer.toString(hex) + 0x 676.00 ns/op 434.00 ns/op 1.56
Object access 1 prop 0.29400 ns/op 0.17700 ns/op 1.66
Map access 1 prop 0.26300 ns/op 0.16700 ns/op 1.57
Object get x1000 10.526 ns/op 6.8720 ns/op 1.53
Map get x1000 0.77300 ns/op 0.64100 ns/op 1.21
Object set x1000 88.643 ns/op 55.437 ns/op 1.60
Map set x1000 75.650 ns/op 50.303 ns/op 1.50
Return object 10000 times 0.33920 ns/op 0.25990 ns/op 1.31
Throw Error 10000 times 6.8898 us/op 4.4222 us/op 1.56
fastMsgIdFn sha256 / 200 bytes 5.0210 us/op 3.6660 us/op 1.37
fastMsgIdFn h32 xxhash / 200 bytes 508.00 ns/op 328.00 ns/op 1.55
fastMsgIdFn h64 xxhash / 200 bytes 761.00 ns/op 483.00 ns/op 1.58
fastMsgIdFn sha256 / 1000 bytes 16.393 us/op 12.007 us/op 1.37
fastMsgIdFn h32 xxhash / 1000 bytes 611.00 ns/op 477.00 ns/op 1.28
fastMsgIdFn h64 xxhash / 1000 bytes 754.00 ns/op 561.00 ns/op 1.34
fastMsgIdFn sha256 / 10000 bytes 142.17 us/op 106.75 us/op 1.33
fastMsgIdFn h32 xxhash / 10000 bytes 2.2910 us/op 2.0640 us/op 1.11
fastMsgIdFn h64 xxhash / 10000 bytes 1.6450 us/op 1.4970 us/op 1.10
enrSubnets - fastDeserialize 64 bits 2.4030 us/op 1.6600 us/op 1.45
enrSubnets - ssz BitVector 64 bits 909.00 ns/op 621.00 ns/op 1.46
enrSubnets - fastDeserialize 4 bits 311.00 ns/op 208.00 ns/op 1.50
enrSubnets - ssz BitVector 4 bits 947.00 ns/op 597.00 ns/op 1.59
prioritizePeers score -10:0 att 32-0.1 sync 2-0 169.82 us/op 119.12 us/op 1.43
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 223.94 us/op 165.27 us/op 1.36
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 270.43 us/op 187.51 us/op 1.44
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 487.59 us/op 360.25 us/op 1.35
prioritizePeers score 0:0 att 64-1 sync 4-1 588.28 us/op 407.72 us/op 1.44
array of 16000 items push then shift 2.2324 us/op 1.6841 us/op 1.33
LinkedList of 16000 items push then shift 11.168 ns/op 9.0970 ns/op 1.23
array of 16000 items push then pop 144.13 ns/op 104.91 ns/op 1.37
LinkedList of 16000 items push then pop 11.105 ns/op 8.7770 ns/op 1.27
array of 24000 items push then shift 3.2866 us/op 2.9357 us/op 1.12
LinkedList of 24000 items push then shift 10.993 ns/op 9.8650 ns/op 1.11
array of 24000 items push then pop 114.11 ns/op 84.637 ns/op 1.35
LinkedList of 24000 items push then pop 11.007 ns/op 8.7950 ns/op 1.25
intersect bitArray bitLen 8 23.047 ns/op 13.585 ns/op 1.70
intersect array and set length 8 137.07 ns/op 80.190 ns/op 1.71
intersect bitArray bitLen 128 61.757 ns/op 44.500 ns/op 1.39
intersect array and set length 128 1.7678 us/op 1.0692 us/op 1.65
Buffer.concat 32 items 5.0150 us/op 2.6610 us/op 1.88
Uint8Array.set 32 items 4.8340 us/op 2.3070 us/op 2.10
transfer serialized Status (84 B) 3.7080 us/op 2.2060 us/op 1.68
copy serialized Status (84 B) 2.9350 us/op 1.8080 us/op 1.62
transfer serialized SignedVoluntaryExit (112 B) 3.6440 us/op 2.1940 us/op 1.66
copy serialized SignedVoluntaryExit (112 B) 3.0340 us/op 1.7580 us/op 1.73
transfer serialized ProposerSlashing (416 B) 4.7600 us/op 2.2570 us/op 2.11
copy serialized ProposerSlashing (416 B) 5.2430 us/op 2.5620 us/op 2.05
transfer serialized Attestation (485 B) 5.2370 us/op 2.4250 us/op 2.16
copy serialized Attestation (485 B) 4.1720 us/op 2.1960 us/op 1.90
transfer serialized AttesterSlashing (33232 B) 5.1600 us/op 2.5910 us/op 1.99
copy serialized AttesterSlashing (33232 B) 11.871 us/op 5.4750 us/op 2.17
transfer serialized Small SignedBeaconBlock (128000 B) 5.0550 us/op 2.6240 us/op 1.93
copy serialized Small SignedBeaconBlock (128000 B) 29.389 us/op 25.130 us/op 1.17
transfer serialized Avg SignedBeaconBlock (200000 B) 5.4840 us/op 2.9200 us/op 1.88
copy serialized Avg SignedBeaconBlock (200000 B) 47.053 us/op 19.414 us/op 2.42
transfer serialized BlobsSidecar (524380 B) 6.3830 us/op 3.0530 us/op 2.09
copy serialized BlobsSidecar (524380 B) 110.55 us/op 172.64 us/op 0.64
transfer serialized Big SignedBeaconBlock (1000000 B) 6.9530 us/op 3.5840 us/op 1.94
copy serialized Big SignedBeaconBlock (1000000 B) 195.24 us/op 408.04 us/op 0.48
pass gossip attestations to forkchoice per slot 3.9042 ms/op 2.8627 ms/op 1.36
forkChoice updateHead vc 100000 bc 64 eq 0 2.8696 ms/op 2.1602 ms/op 1.33
forkChoice updateHead vc 600000 bc 64 eq 0 17.677 ms/op 11.991 ms/op 1.47
forkChoice updateHead vc 1000000 bc 64 eq 0 29.237 ms/op 24.120 ms/op 1.21
forkChoice updateHead vc 600000 bc 320 eq 0 25.057 ms/op 18.114 ms/op 1.38
forkChoice updateHead vc 600000 bc 1200 eq 0 129.15 ms/op 90.750 ms/op 1.42
forkChoice updateHead vc 600000 bc 64 eq 1000 29.212 ms/op 21.351 ms/op 1.37
forkChoice updateHead vc 600000 bc 64 eq 10000 30.855 ms/op 23.501 ms/op 1.31
forkChoice updateHead vc 600000 bc 64 eq 300000 47.205 ms/op 35.740 ms/op 1.32
computeDeltas 4.2907 ms/op 3.1849 ms/op 1.35
computeProposerBoostScoreFromBalances 2.4031 ms/op 1.8692 ms/op 1.29
altair processAttestation - 250000 vs - 7PWei normalcase 3.7047 ms/op 3.3574 ms/op 1.10
altair processAttestation - 250000 vs - 7PWei worstcase 5.5319 ms/op 5.4091 ms/op 1.02
altair processAttestation - setStatus - 1/6 committees join 197.10 us/op 146.55 us/op 1.34
altair processAttestation - setStatus - 1/3 committees join 387.67 us/op 290.94 us/op 1.33
altair processAttestation - setStatus - 1/2 committees join 522.04 us/op 391.16 us/op 1.33
altair processAttestation - setStatus - 2/3 committees join 668.62 us/op 501.00 us/op 1.33
altair processAttestation - setStatus - 4/5 committees join 925.98 us/op 684.47 us/op 1.35
altair processAttestation - setStatus - 100% committees join 1.0840 ms/op 826.57 us/op 1.31
altair processBlock - 250000 vs - 7PWei normalcase 22.580 ms/op 19.046 ms/op 1.19
altair processBlock - 250000 vs - 7PWei normalcase hashState 37.265 ms/op 27.782 ms/op 1.34
altair processBlock - 250000 vs - 7PWei worstcase 62.655 ms/op 52.028 ms/op 1.20
altair processBlock - 250000 vs - 7PWei worstcase hashState 98.067 ms/op 75.285 ms/op 1.30
phase0 processBlock - 250000 vs - 7PWei normalcase 3.4004 ms/op 2.0950 ms/op 1.62
phase0 processBlock - 250000 vs - 7PWei worstcase 41.065 ms/op 29.819 ms/op 1.38
altair processEth1Data - 250000 vs - 7PWei normalcase 756.85 us/op 494.12 us/op 1.53
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 11.496 us/op 8.8080 us/op 1.31
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 25.066 us/op 29.426 us/op 0.85
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 14.880 us/op 10.315 us/op 1.44
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 16.041 us/op 7.9950 us/op 2.01
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 142.79 us/op 102.06 us/op 1.40
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 887.99 us/op 663.92 us/op 1.34
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.2649 ms/op 921.27 us/op 1.37
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.2181 ms/op 904.26 us/op 1.35
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 4.3021 ms/op 2.9387 ms/op 1.46
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.5408 ms/op 1.7940 ms/op 1.42
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 7.4941 ms/op 3.9788 ms/op 1.88
Tree 40 250000 create 793.08 ms/op 344.53 ms/op 2.30
Tree 40 250000 get(125000) 290.88 ns/op 191.65 ns/op 1.52
Tree 40 250000 set(125000) 2.7255 us/op 1.0161 us/op 2.68
Tree 40 250000 toArray() 31.660 ms/op 20.483 ms/op 1.55
Tree 40 250000 iterate all - toArray() + loop 31.968 ms/op 21.051 ms/op 1.52
Tree 40 250000 iterate all - get(i) 113.00 ms/op 73.311 ms/op 1.54
MutableVector 250000 create 14.877 ms/op 10.942 ms/op 1.36
MutableVector 250000 get(125000) 9.5890 ns/op 6.3280 ns/op 1.52
MutableVector 250000 set(125000) 706.74 ns/op 274.73 ns/op 2.57
MutableVector 250000 toArray() 4.9149 ms/op 3.0310 ms/op 1.62
MutableVector 250000 iterate all - toArray() + loop 5.0995 ms/op 3.0173 ms/op 1.69
MutableVector 250000 iterate all - get(i) 2.4291 ms/op 1.5336 ms/op 1.58
Array 250000 create 4.8204 ms/op 2.9725 ms/op 1.62
Array 250000 clone - spread 4.8093 ms/op 1.2097 ms/op 3.98
Array 250000 get(125000) 1.7860 ns/op 0.56900 ns/op 3.14
Array 250000 set(125000) 1.9930 ns/op 0.64600 ns/op 3.09
Array 250000 iterate all - loop 115.94 us/op 108.63 us/op 1.07
effectiveBalanceIncrements clone Uint8Array 300000 77.084 us/op 39.408 us/op 1.96
effectiveBalanceIncrements clone MutableVector 300000 1.7410 us/op 328.00 ns/op 5.31
effectiveBalanceIncrements rw all Uint8Array 300000 266.64 us/op 169.95 us/op 1.57
effectiveBalanceIncrements rw all MutableVector 300000 205.48 ms/op 81.113 ms/op 2.53
phase0 afterProcessEpoch - 250000 vs - 7PWei 167.08 ms/op 116.48 ms/op 1.43
phase0 beforeProcessEpoch - 250000 vs - 7PWei 46.650 ms/op 35.667 ms/op 1.31
altair processEpoch - mainnet_e81889 490.92 ms/op 322.83 ms/op 1.52
mainnet_e81889 - altair beforeProcessEpoch 75.777 ms/op 55.553 ms/op 1.36
mainnet_e81889 - altair processJustificationAndFinalization 22.893 us/op 21.774 us/op 1.05
mainnet_e81889 - altair processInactivityUpdates 9.5683 ms/op 6.5549 ms/op 1.46
mainnet_e81889 - altair processRewardsAndPenalties 87.238 ms/op 67.200 ms/op 1.30
mainnet_e81889 - altair processRegistryUpdates 5.3890 us/op 2.9790 us/op 1.81
mainnet_e81889 - altair processSlashings 1.0480 us/op 739.00 ns/op 1.42
mainnet_e81889 - altair processEth1DataReset 1.2480 us/op 563.00 ns/op 2.22
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.6691 ms/op 1.3608 ms/op 1.23
mainnet_e81889 - altair processSlashingsReset 11.948 us/op 6.8390 us/op 1.75
mainnet_e81889 - altair processRandaoMixesReset 10.242 us/op 8.5000 us/op 1.20
mainnet_e81889 - altair processHistoricalRootsUpdate 1.2300 us/op 1.0710 us/op 1.15
mainnet_e81889 - altair processParticipationFlagUpdates 5.1940 us/op 3.7490 us/op 1.39
mainnet_e81889 - altair processSyncCommitteeUpdates 1.0170 us/op 745.00 ns/op 1.37
mainnet_e81889 - altair afterProcessEpoch 191.02 ms/op 137.68 ms/op 1.39
phase0 processEpoch - mainnet_e58758 521.19 ms/op 363.05 ms/op 1.44
mainnet_e58758 - phase0 beforeProcessEpoch 155.92 ms/op 144.01 ms/op 1.08
mainnet_e58758 - phase0 processJustificationAndFinalization 30.471 us/op 18.615 us/op 1.64
mainnet_e58758 - phase0 processRewardsAndPenalties 81.611 ms/op 63.198 ms/op 1.29
mainnet_e58758 - phase0 processRegistryUpdates 13.833 us/op 8.9600 us/op 1.54
mainnet_e58758 - phase0 processSlashings 669.00 ns/op 503.00 ns/op 1.33
mainnet_e58758 - phase0 processEth1DataReset 1.0670 us/op 617.00 ns/op 1.73
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.3512 ms/op 997.85 us/op 1.35
mainnet_e58758 - phase0 processSlashingsReset 6.2830 us/op 3.4060 us/op 1.84
mainnet_e58758 - phase0 processRandaoMixesReset 8.9970 us/op 4.7310 us/op 1.90
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.1700 us/op 634.00 ns/op 1.85
mainnet_e58758 - phase0 processParticipationRecordUpdates 7.4220 us/op 3.8610 us/op 1.92
mainnet_e58758 - phase0 afterProcessEpoch 147.05 ms/op 98.701 ms/op 1.49
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.7040 ms/op 1.2435 ms/op 1.37
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.9854 ms/op 1.5140 ms/op 1.31
altair processInactivityUpdates - 250000 normalcase 28.404 ms/op 25.295 ms/op 1.12
altair processInactivityUpdates - 250000 worstcase 30.900 ms/op 26.993 ms/op 1.14
phase0 processRegistryUpdates - 250000 normalcase 11.292 us/op 7.8690 us/op 1.43
phase0 processRegistryUpdates - 250000 badcase_full_deposits 384.31 us/op 275.08 us/op 1.40
phase0 processRegistryUpdates - 250000 worstcase 0.5 169.61 ms/op 123.65 ms/op 1.37
altair processRewardsAndPenalties - 250000 normalcase 80.884 ms/op 67.505 ms/op 1.20
altair processRewardsAndPenalties - 250000 worstcase 85.685 ms/op 72.977 ms/op 1.17
phase0 getAttestationDeltas - 250000 normalcase 8.9593 ms/op 8.4617 ms/op 1.06
phase0 getAttestationDeltas - 250000 worstcase 12.351 ms/op 7.8763 ms/op 1.57
phase0 processSlashings - 250000 worstcase 5.0366 ms/op 3.6702 ms/op 1.37
altair processSyncCommitteeUpdates - 250000 293.70 ms/op 190.42 ms/op 1.54
BeaconState.hashTreeRoot - No change 458.00 ns/op 351.00 ns/op 1.30
BeaconState.hashTreeRoot - 1 full validator 68.358 us/op 56.325 us/op 1.21
BeaconState.hashTreeRoot - 32 full validator 687.43 us/op 543.57 us/op 1.26
BeaconState.hashTreeRoot - 512 full validator 8.0541 ms/op 5.3411 ms/op 1.51
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 85.982 us/op 63.252 us/op 1.36
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2427 ms/op 1.0573 ms/op 1.18
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.446 ms/op 13.200 ms/op 1.25
BeaconState.hashTreeRoot - 1 balances 70.106 us/op 54.912 us/op 1.28
BeaconState.hashTreeRoot - 32 balances 627.62 us/op 470.56 us/op 1.33
BeaconState.hashTreeRoot - 512 balances 6.1141 ms/op 5.3396 ms/op 1.15
BeaconState.hashTreeRoot - 250000 balances 103.57 ms/op 81.521 ms/op 1.27
aggregationBits - 2048 els - zipIndexesInBitList 34.531 us/op 17.884 us/op 1.93
regular array get 100000 times 45.407 us/op 34.115 us/op 1.33
wrappedArray get 100000 times 45.562 us/op 44.305 us/op 1.03
arrayWithProxy get 100000 times 34.238 ms/op 15.810 ms/op 2.17
ssz.Root.equals 947.00 ns/op 547.00 ns/op 1.73
byteArrayEquals 952.00 ns/op 543.00 ns/op 1.75
shuffle list - 16384 els 10.475 ms/op 7.0243 ms/op 1.49
shuffle list - 250000 els 150.41 ms/op 103.64 ms/op 1.45
processSlot - 1 slots 14.209 us/op 10.126 us/op 1.40
processSlot - 32 slots 2.1279 ms/op 1.4016 ms/op 1.52
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 48.546 ms/op 40.427 ms/op 1.20
getCommitteeAssignments - req 1 vs - 250000 vc 4.2257 ms/op 3.3053 ms/op 1.28
getCommitteeAssignments - req 100 vs - 250000 vc 5.9971 ms/op 4.3948 ms/op 1.36
getCommitteeAssignments - req 1000 vs - 250000 vc 6.4907 ms/op 4.9698 ms/op 1.31
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 7.9600 ns/op 5.3300 ns/op 1.49
state getBlockRootAtSlot - 250000 vs - 7PWei 1.5262 us/op 860.90 ns/op 1.77
computeProposers - vc 250000 18.287 ms/op 11.300 ms/op 1.62
computeEpochShuffling - vc 250000 153.58 ms/op 111.83 ms/op 1.37
getNextSyncCommittee - vc 250000 299.36 ms/op 195.45 ms/op 1.53
computeSigningRoot for AttestationData 20.389 us/op 15.694 us/op 1.30
hash AttestationData serialized data then Buffer.toString(base64) 3.5561 us/op 2.7423 us/op 1.30
toHexString serialized data 1.9233 us/op 1.6484 us/op 1.17
Buffer.toString(base64) 591.44 ns/op 383.83 ns/op 1.54

by benchmarkbot/action

@wemeetagain wemeetagain merged commit 8337608 into unstable Jun 22, 2023
@wemeetagain wemeetagain deleted the cayman/update-prometheus-gc-stats branch June 22, 2023 16:19
@wemeetagain wemeetagain mentioned this pull request Jun 23, 2023
4 tasks
@wemeetagain
Copy link
Member Author

🎉 This PR is included in v1.10.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants