Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix nightly release workflow #3637

Merged
merged 1 commit into from
Jan 18, 2022
Merged

Fix nightly release workflow #3637

merged 1 commit into from
Jan 18, 2022

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Jan 18, 2022

Motivation
A previous PR #3601 which aimed to fix how the nightly --canary versions were being tagged, used from-package in lerna publish. However it was discovered that using from-package flag totatly ignore any other options with lerna comparing the published versions against package.json's version.

This PR restores the original lerna publish --canary... and adds comments as to how to make sure it produces correct publishing tag.
Description
Lerna publish ignores what are called lightweight or commit tags (do git for-each-ref refs/tags to get tag type). It only takes into care the proper annotated tags, which can be produced by git tag -a option, or better through lerna publish. It seems like, the tags post 0.32 are light weight:

$ git for-each-ref refs/tags |grep -vE commit
d345c93b9a7afd8eeeb378f51450a90039084dcd tag	refs/tags/v0.13.0
325b1116f97a11f86763a7490c62d5565535875a tag	refs/tags/v0.14.0
f1918f8ca7eeda595225ad00432f68fcfb1134f4 tag	refs/tags/v0.15.0
b3036a20f5f173733574cb00a16aeec96cfd294d tag	refs/tags/v0.16.0
34d1677d0a3eeb0e82579f7c3e40504b19f20aa4 tag	refs/tags/v0.17.0
249ce56e504693e9482348bc4d0b745c9de1a3cb tag	refs/tags/v0.18.0
35f8c997730b3c0101bcf25b805af496014e0b26 tag	refs/tags/v0.19.0
65bbeedb8dbdd9943831847cfcecd2d2911e859f tag	refs/tags/v0.20.0
ead96ad89dd45b3f16ed6552d590452e2c3458a0 tag	refs/tags/v0.21.0
139cf9550c0fa9909c2586d76dd679a9f53e0d80 tag	refs/tags/v0.23.0
b7d8429fab4e59a60acb99bc0d918dc6d4dd4bbd tag	refs/tags/v0.24.0
39142e4573d91497ab0887943bec20f16152b73e tag	refs/tags/v0.24.1
31fdc2c2c7a737bb6e8a4dfa3c166cb9ce3b7634 tag	refs/tags/v0.24.2
dc68d3fcb5ab360b34977d67e74b95200df20138 tag	refs/tags/v0.24.3
7848e77799f2d80948e400206b1f3eb4fbc5444c tag	refs/tags/v0.25.0
ca1c76c9c38c53976efe55025eb3b7617d8ac2f4 tag	refs/tags/v0.25.1
e62779bfa974903561f6ad64ebe1cf6f18c609f3 tag	refs/tags/v0.26.0
3f7f1bb628e79c8ebf7968d92f3f5f346e5be99e tag	refs/tags/v0.27.0
1d791c2197b2f2b85b3a332be6e86f208c798a5a tag	refs/tags/v0.28.0
3228b2966ce3cda8a1ddb937f3bf581782425c3d tag	refs/tags/v0.28.1
a78d0fd62484d0544f6e58367eeeb081265bf454 tag	refs/tags/v0.29.0
4696395abe262b6847ce58cfc84683de8d5a2559 tag	refs/tags/v0.29.1
b6fadd8f0867f8af6e2bd5d2a52b68ab2d65c05e tag	refs/tags/v0.29.2
30ac87b6a73bffccb1b2254789b5990b92222082 tag	refs/tags/v0.29.3
b54d1661713c816e390442c48cd1b4672f06c6d1 tag	refs/tags/v0.30.0
220a13d5c20770a3cb98b32704d418decd2bee2c tag	refs/tags/v0.31.0
f022b80b92fd7e96f75e474615191c94f347451c tag	refs/tags/v0.32.0

and lerna publish --canarary gives these tags as 0.32.0 is the last proper annotated tag:

deploy@r9core12:/mnt/code/lodestar/master$ yarn lerna publish --canary --force-publish
yarn run v1.22.11
$ lerna publish --canary --force-publish
lerna notice cli v4.0.0
lerna info canary enabled
lerna WARN Yarn's registry proxy is broken, replacing with public npm registry
lerna WARN If you don't have an npm token, you should exit and run `npm login`
lerna WARN force-publish all packages
lerna info Assuming all packages changed

Found 13 packages to publish:
 - @chainsafe/lodestar-api => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar-beacon-state-transition => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar-cli => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar-config => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar-db => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar-fork-choice => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar-light-client => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar-params => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar-spec-test-util => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar-types => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar-utils => 0.32.1-alpha.93+94e67f6b6af
 - @chainsafe/lodestar-validator => 0.32.1-alpha.93+94e67f6b6af

So original lerna publish --canary --force-publish will work correctly with the correct tagging.

@codecov
Copy link

codecov bot commented Jan 18, 2022

Codecov Report

Merging #3637 (94e67f6) into master (5021f02) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3637   +/-   ##
=======================================
  Coverage   37.18%   37.18%           
=======================================
  Files         320      320           
  Lines        8629     8629           
  Branches     1336     1336           
=======================================
  Hits         3209     3209           
  Misses       5272     5272           
  Partials      148      148           

@codeclimate
Copy link

codeclimate bot commented Jan 18, 2022

Code Climate has analyzed commit 94e67f6 and detected 0 issues on this pull request.

View more on Code Climate.

@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: ae703d7 Previous: 5021f02 Ratio
BeaconState.hashTreeRoot - No change 684.00 ns/op 610.00 ns/op 1.12
BeaconState.hashTreeRoot - 1 full validator 158.54 us/op 142.06 us/op 1.12
BeaconState.hashTreeRoot - 32 full validator 2.3488 ms/op 2.1681 ms/op 1.08
BeaconState.hashTreeRoot - 512 full validator 32.072 ms/op 27.975 ms/op 1.15
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 156.19 us/op 142.55 us/op 1.10
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.5189 ms/op 2.3049 ms/op 1.09
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 35.245 ms/op 30.924 ms/op 1.14
BeaconState.hashTreeRoot - 1 balances 114.90 us/op 102.08 us/op 1.13
BeaconState.hashTreeRoot - 32 balances 914.27 us/op 845.13 us/op 1.08
BeaconState.hashTreeRoot - 512 balances 9.4519 ms/op 8.0956 ms/op 1.17
BeaconState.hashTreeRoot - 250000 balances 181.76 ms/op 158.77 ms/op 1.14
processSlot - 1 slots 60.985 us/op 137.60 us/op 0.44
processSlot - 32 slots 3.6370 ms/op 3.3817 ms/op 1.08
getCommitteeAssignments - req 1 vs - 250000 vc 6.5962 ms/op 5.9385 ms/op 1.11
getCommitteeAssignments - req 100 vs - 250000 vc 9.1193 ms/op 7.8948 ms/op 1.16
getCommitteeAssignments - req 1000 vs - 250000 vc 9.5577 ms/op 8.4989 ms/op 1.12
computeProposers - vc 250000 25.334 ms/op 22.448 ms/op 1.13
computeEpochShuffling - vc 250000 222.69 ms/op 198.64 ms/op 1.12
getNextSyncCommittee - vc 250000 412.83 ms/op 374.11 ms/op 1.10
altair processAttestation - 250000 vs - 7PWei normalcase 53.585 ms/op 46.903 ms/op 1.14
altair processAttestation - 250000 vs - 7PWei worstcase 56.172 ms/op 49.187 ms/op 1.14
altair processAttestation - setStatus - 1/6 committees join 15.543 ms/op 11.635 ms/op 1.34
altair processAttestation - setStatus - 1/3 committees join 29.573 ms/op 24.709 ms/op 1.20
altair processAttestation - setStatus - 1/2 committees join 43.383 ms/op 39.409 ms/op 1.10
altair processAttestation - setStatus - 2/3 committees join 60.097 ms/op 49.216 ms/op 1.22
altair processAttestation - setStatus - 4/5 committees join 72.722 ms/op 57.372 ms/op 1.27
altair processAttestation - setStatus - 100% committees join 97.588 ms/op 74.645 ms/op 1.31
altair processAttestation - updateEpochParticipants - 1/6 committees join 15.716 ms/op 13.124 ms/op 1.20
altair processAttestation - updateEpochParticipants - 1/3 committees join 37.200 ms/op 23.243 ms/op 1.60
altair processAttestation - updateEpochParticipants - 1/2 committees join 35.465 ms/op 34.075 ms/op 1.04
altair processAttestation - updateEpochParticipants - 2/3 committees join 38.080 ms/op 28.728 ms/op 1.33
altair processAttestation - updateEpochParticipants - 4/5 committees join 48.157 ms/op 32.233 ms/op 1.49
altair processAttestation - updateEpochParticipants - 100% committees join 40.694 ms/op 32.695 ms/op 1.24
altair processAttestation - updateAllStatus 31.478 ms/op 24.140 ms/op 1.30
altair processBlock - 250000 vs - 7PWei normalcase 55.463 ms/op 57.724 ms/op 0.96
altair processBlock - 250000 vs - 7PWei worstcase 144.93 ms/op 128.95 ms/op 1.12
altair processEpoch - mainnet_e81889 1.2927 s/op 1.1234 s/op 1.15
mainnet_e81889 - altair beforeProcessEpoch 338.63 ms/op 383.19 ms/op 0.88
mainnet_e81889 - altair processJustificationAndFinalization 72.064 us/op 66.755 us/op 1.08
mainnet_e81889 - altair processInactivityUpdates 22.784 ms/op 19.744 ms/op 1.15
mainnet_e81889 - altair processRewardsAndPenalties 252.10 ms/op 156.11 ms/op 1.61
mainnet_e81889 - altair processRegistryUpdates 11.124 us/op 15.759 us/op 0.71
mainnet_e81889 - altair processSlashings 3.3200 us/op 3.2900 us/op 1.01
mainnet_e81889 - altair processEth1DataReset 3.5460 us/op 3.0250 us/op 1.17
mainnet_e81889 - altair processEffectiveBalanceUpdates 15.441 ms/op 13.404 ms/op 1.15
mainnet_e81889 - altair processSlashingsReset 18.249 us/op 23.159 us/op 0.79
mainnet_e81889 - altair processRandaoMixesReset 23.264 us/op 30.711 us/op 0.76
mainnet_e81889 - altair processHistoricalRootsUpdate 4.5330 us/op 6.4000 us/op 0.71
mainnet_e81889 - altair processParticipationFlagUpdates 131.82 ms/op 115.49 ms/op 1.14
mainnet_e81889 - altair processSyncCommitteeUpdates 2.8120 us/op 4.2160 us/op 0.67
mainnet_e81889 - altair afterProcessEpoch 278.52 ms/op 285.47 ms/op 0.98
altair processInactivityUpdates - 250000 normalcase 90.949 ms/op 85.113 ms/op 1.07
altair processInactivityUpdates - 250000 worstcase 94.440 ms/op 76.095 ms/op 1.24
altair processParticipationFlagUpdates - 250000 anycase 196.64 ms/op 120.21 ms/op 1.64
altair processRewardsAndPenalties - 250000 normalcase 157.93 ms/op 164.25 ms/op 0.96
altair processRewardsAndPenalties - 250000 worstcase 157.48 ms/op 165.14 ms/op 0.95
altair processSyncCommitteeUpdates - 250000 446.58 ms/op 395.50 ms/op 1.13
Tree 40 250000 create 1.0536 s/op 869.82 ms/op 1.21
Tree 40 250000 get(125000) 401.99 ns/op 356.69 ns/op 1.13
Tree 40 250000 set(125000) 2.4373 us/op 1.8459 us/op 1.32
Tree 40 250000 toArray() 47.236 ms/op 39.329 ms/op 1.20
Tree 40 250000 iterate all - toArray() + loop 54.399 ms/op 44.405 ms/op 1.23
Tree 40 250000 iterate all - get(i) 146.62 ms/op 126.60 ms/op 1.16
MutableVector 250000 create 32.295 ms/op 21.206 ms/op 1.52
MutableVector 250000 get(125000) 16.890 ns/op 15.965 ns/op 1.06
MutableVector 250000 set(125000) 705.22 ns/op 464.69 ns/op 1.52
MutableVector 250000 toArray() 11.241 ms/op 9.0036 ms/op 1.25
MutableVector 250000 iterate all - toArray() + loop 11.542 ms/op 21.562 ms/op 0.54
MutableVector 250000 iterate all - get(i) 4.2900 ms/op 3.4998 ms/op 1.23
Array 250000 create 7.3955 ms/op 5.9029 ms/op 1.25
Array 250000 clone - spread 3.1164 ms/op 1.9304 ms/op 1.61
Array 250000 get(125000) 1.3960 ns/op 0.96700 ns/op 1.44
Array 250000 set(125000) 1.4350 ns/op 0.97400 ns/op 1.47
Array 250000 iterate all - loop 204.84 us/op 199.15 us/op 1.03
aggregationBits - 2048 els - readonlyValues 367.77 us/op 237.72 us/op 1.55
aggregationBits - 2048 els - zipIndexesInBitList 57.055 us/op 41.191 us/op 1.39
regular array get 100000 times 84.216 us/op 77.332 us/op 1.09
wrappedArray get 100000 times 83.628 us/op 78.893 us/op 1.06
arrayWithProxy get 100000 times 37.646 ms/op 33.978 ms/op 1.11
ssz.Root.equals 1.5390 us/op 1.2280 us/op 1.25
ssz.Root.equals with valueOf() 1.9300 us/op 1.3410 us/op 1.44
byteArrayEquals with valueOf() 1.8420 us/op 1.3550 us/op 1.36
phase0 processBlock - 250000 vs - 7PWei normalcase 13.889 ms/op 11.566 ms/op 1.20
phase0 processBlock - 250000 vs - 7PWei worstcase 94.765 ms/op 84.789 ms/op 1.12
phase0 afterProcessEpoch - 250000 vs - 7PWei 251.94 ms/op 228.85 ms/op 1.10
phase0 beforeProcessEpoch - 250000 vs - 7PWei 726.17 ms/op 584.70 ms/op 1.24
phase0 processEpoch - mainnet_e58758 953.37 ms/op 846.25 ms/op 1.13
mainnet_e58758 - phase0 beforeProcessEpoch 568.54 ms/op 445.97 ms/op 1.27
mainnet_e58758 - phase0 processJustificationAndFinalization 70.631 us/op 91.359 us/op 0.77
mainnet_e58758 - phase0 processRewardsAndPenalties 153.78 ms/op 107.59 ms/op 1.43
mainnet_e58758 - phase0 processRegistryUpdates 96.810 us/op 74.427 us/op 1.30
mainnet_e58758 - phase0 processSlashings 3.5310 us/op 5.0990 us/op 0.69
mainnet_e58758 - phase0 processEth1DataReset 3.0620 us/op 4.1370 us/op 0.74
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 12.847 ms/op 10.553 ms/op 1.22
mainnet_e58758 - phase0 processSlashingsReset 15.042 us/op 21.266 us/op 0.71
mainnet_e58758 - phase0 processRandaoMixesReset 20.093 us/op 26.753 us/op 0.75
mainnet_e58758 - phase0 processHistoricalRootsUpdate 4.3370 us/op 4.8380 us/op 0.90
mainnet_e58758 - phase0 processParticipationRecordUpdates 14.974 us/op 15.857 us/op 0.94
mainnet_e58758 - phase0 afterProcessEpoch 222.39 ms/op 214.10 ms/op 1.04
phase0 processEffectiveBalanceUpdates - 250000 normalcase 14.201 ms/op 12.868 ms/op 1.10
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.6623 s/op 1.4988 s/op 1.11
phase0 processRegistryUpdates - 250000 normalcase 59.490 us/op 71.265 us/op 0.83
phase0 processRegistryUpdates - 250000 badcase_full_deposits 3.3874 ms/op 3.1490 ms/op 1.08
phase0 processRegistryUpdates - 250000 worstcase 0.5 2.1239 s/op 1.8859 s/op 1.13
phase0 getAttestationDeltas - 250000 normalcase 42.831 ms/op 37.424 ms/op 1.14
phase0 getAttestationDeltas - 250000 worstcase 41.686 ms/op 38.704 ms/op 1.08
phase0 processSlashings - 250000 worstcase 45.270 ms/op 38.206 ms/op 1.18
shuffle list - 16384 els 16.172 ms/op 16.739 ms/op 0.97
shuffle list - 250000 els 230.16 ms/op 198.49 ms/op 1.16
getEffectiveBalances - 250000 vs - 7PWei 13.932 ms/op 10.718 ms/op 1.30
pass gossip attestations to forkchoice per slot 17.121 ms/op 15.872 ms/op 1.08
computeDeltas 3.8081 ms/op 3.9626 ms/op 0.96
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.9751 ms/op 2.0258 ms/op 1.47
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 770.33 us/op 735.99 us/op 1.05
BLS verify - blst-native 2.2262 ms/op 2.1109 ms/op 1.05
BLS verifyMultipleSignatures 3 - blst-native 4.5794 ms/op 4.3167 ms/op 1.06
BLS verifyMultipleSignatures 8 - blst-native 9.8961 ms/op 9.1429 ms/op 1.08
BLS verifyMultipleSignatures 32 - blst-native 35.865 ms/op 34.234 ms/op 1.05
BLS aggregatePubkeys 32 - blst-native 47.575 us/op 44.187 us/op 1.08
BLS aggregatePubkeys 128 - blst-native 184.18 us/op 172.02 us/op 1.07
getAttestationsForBlock 96.240 ms/op 83.615 ms/op 1.15
CheckpointStateCache - add get delete 22.840 us/op 21.286 us/op 1.07
validate gossip signedAggregateAndProof - struct 5.6422 ms/op 4.9756 ms/op 1.13
validate gossip signedAggregateAndProof - treeBacked 5.5286 ms/op 5.0422 ms/op 1.10
validate gossip attestation - struct 2.5686 ms/op 2.4548 ms/op 1.05
validate gossip attestation - treeBacked 2.6573 ms/op 2.5597 ms/op 1.04
bytes32 toHexString 2.1390 us/op 2.0070 us/op 1.07
bytes32 Buffer.toString(hex) 861.00 ns/op 783.00 ns/op 1.10
bytes32 Buffer.toString(hex) from Uint8Array 1.0720 us/op 1.0900 us/op 0.98
bytes32 Buffer.toString(hex) + 0x 861.00 ns/op 808.00 ns/op 1.07
Object access 1 prop 0.40000 ns/op 0.38200 ns/op 1.05
Map access 1 prop 0.36000 ns/op 0.31500 ns/op 1.14
Object get x1000 21.329 ns/op 20.254 ns/op 1.05
Map get x1000 1.1900 ns/op 1.2090 ns/op 0.98
Object set x1000 131.02 ns/op 124.18 ns/op 1.06
Map set x1000 80.116 ns/op 69.561 ns/op 1.15
Return object 10000 times 0.45540 ns/op 0.41310 ns/op 1.10
Throw Error 10000 times 7.4024 us/op 6.5060 us/op 1.14
enrSubnets - fastDeserialize 64 bits 1.4950 us/op 1.3280 us/op 1.13
enrSubnets - ssz BitVector 64 bits 20.425 us/op 17.973 us/op 1.14
enrSubnets - fastDeserialize 4 bits 548.00 ns/op 508.00 ns/op 1.08
enrSubnets - ssz BitVector 4 bits 3.5250 us/op 3.1880 us/op 1.11
RateTracker 1000000 limit, 1 obj count per request 221.43 ns/op 191.68 ns/op 1.16
RateTracker 1000000 limit, 2 obj count per request 165.51 ns/op 154.51 ns/op 1.07
RateTracker 1000000 limit, 4 obj count per request 136.56 ns/op 120.90 ns/op 1.13
RateTracker 1000000 limit, 8 obj count per request 127.85 ns/op 109.70 ns/op 1.17
RateTracker with prune 4.7750 us/op 4.3120 us/op 1.11

by benchmarkbot/action

Copy link
Contributor

@dapplion dapplion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lerna is more confusing than a proof of stake protocol lol let's see if this works

@dapplion dapplion merged commit 9be6d13 into master Jan 18, 2022
@dapplion dapplion deleted the g11tech/nightlyfix branch January 18, 2022 19:54
@dapplion dapplion mentioned this pull request Jul 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants