Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

New slots/auctions architecture #2294

Merged
merged 173 commits into from
Mar 14, 2021
Merged

New slots/auctions architecture #2294

merged 173 commits into from
Mar 14, 2021

Conversation

gavofyork
Copy link
Member

@gavofyork gavofyork commented Jan 20, 2021

Refactor the slots/leases and auctions code

  • split Slots into two pallets - Slots (which handles leases) and Auctions, separated by a trait.
  • require ParaIds to be claimed before any bids can be made.
  • allow anyone/thing to bid for any para to gain a parachain slot in the future.
  • allow multiple funders to lease a parachain slot for the same parachain in different periods.
  • use reserve system throughout, rather than burn/mint for the leasing.
  • guarantee that the funds never leave ownership of the leaser.

TODO:

Relies on:

@gavofyork gavofyork changed the title Gav new slots New slots/auctions architecture Jan 20, 2021
@github-actions github-actions bot added the A0-please_review Pull request needs code review. label Jan 20, 2021
runtime/common/src/traits.rs Outdated Show resolved Hide resolved
runtime/common/src/traits.rs Outdated Show resolved Hide resolved
runtime/common/src/slots.rs Outdated Show resolved Hide resolved
runtime/common/src/slots.rs Outdated Show resolved Hide resolved
gavofyork and others added 4 commits January 28, 2021 17:20
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
@parity-benchapp
Copy link

parity-benchapp bot commented Mar 14, 2021

Finished benchmark for branch: gav-new-slots

Benchmark: Benchmark Runtime Westend Pallet

cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Results

Pallet: "crowdloan", Extrinsic: "create", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis

-- Extrinsic Time --

Model:
Time ~= 77.55
µs

Reads = 3
Writes = 2
Min Squares Analysis

-- Extrinsic Time --

Model:
Time ~= 77.55
µs

Reads = 3
Writes = 2
Pallet: "crowdloan", Extrinsic: "contribute", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis

-- Extrinsic Time --

Model:
Time ~= 428.9
µs

Reads = 6
Writes = 4
Min Squares Analysis

-- Extrinsic Time --

Model:
Time ~= 428.9
µs

Reads = 6
Writes = 4
Pallet: "crowdloan", Extrinsic: "withdraw", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis

-- Extrinsic Time --

Model:
Time ~= 116.8
µs

Reads = 4
Writes = 4
Min Squares Analysis

-- Extrinsic Time --

Model:
Time ~= 116.8
µs

Reads = 4
Writes = 4
Pallet: "crowdloan", Extrinsic: "dissolve", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis

-- Extrinsic Time --

Model:
Time ~= 116.1
+ k 0.887
µs

Reads = 3 + (0 * k)
Writes = 3 + (1 * k)
Min Squares Analysis

-- Extrinsic Time --

Data points distribution:
k mean µs sigma µs %
0 328.3 3.431 1.0%
2 116.3 0.691 0.5%
4 117.7 0.379 0.3%
6 120.4 0.791 0.6%
8 121.9 0.591 0.4%
10 123 0.248 0.2%
12 126.1 0.63 0.4%
14 127.3 0.384 0.3%
16 129.7 0.769 0.5%
18 132.1 0.441 0.3%
20 133.3 0.762 0.5%
22 135.5 0.431 0.3%
24 137.3 0.876 0.6%
26 139.4 0.943 0.6%
28 140.5 0.771 0.5%
30 142.2 0.632 0.4%
32 144.4 0.751 0.5%
34 147.2 1.398 0.9%
36 148.1 0.633 0.4%
38 150.4 0.827 0.5%
40 152.6 0.479 0.3%
42 153.6 0.714 0.4%
44 156.1 0.723 0.4%
46 158.7 0.732 0.4%
48 160.5 0.926 0.5%
50 160.4 0.408 0.2%
52 162.9 0.8 0.4%
54 166.4 0.559 0.3%
56 167.2 0.957 0.5%
58 168.4 0.559 0.3%
60 170.7 1.011 0.5%
62 171.8 0.571 0.3%
64 173.9 0.676 0.3%
66 176.1 0.579 0.3%
68 177 0.901 0.5%
70 179 1.214 0.6%
72 180.4 0.804 0.4%
74 181.8 0.712 0.3%
76 184.1 0.557 0.3%
78 186.9 0.685 0.3%
80 187.8 1.451 0.7%
82 189.2 1.148 0.6%
84 190.7 0.893 0.4%
86 190.9 0.746 0.3%
88 193.3 0.777 0.4%
90 195.7 0.681 0.3%
92 196 0.657 0.3%
94 198.2 1.088 0.5%
96 200 0.641 0.3%
98 201 0.521 0.2%
100 202.7 0.769 0.3%

Quality and confidence:
param error
k 0.043

Model:
Time ~= 132.1
+ k 0.65
µs

Reads = 3 + (0 * k)
Writes = 3 + (1 * k)
Pallet: "crowdloan", Extrinsic: "on_initialize", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis

-- Extrinsic Time --

Model:
Time ~= 10.96
+ n 106.7
µs

Reads = 5 + (4 * n)
Writes = 3 + (2 * n)
Min Squares Analysis

-- Extrinsic Time --

Data points distribution:
n mean µs sigma µs %
2 234 1.114 0.4%
3 336.1 0.898 0.2%
4 442.4 0.374 0.0%
5 548.4 0.916 0.1%
6 653.8 3.827 0.5%
7 754.7 0.797 0.1%
8 859.6 0.887 0.1%
9 961.1 4.002 0.4%
10 1068 10.42 0.9%
11 1177 8.677 0.7%
12 1286 4.816 0.3%
13 1393 5.8 0.4%
14 1494 2.825 0.1%
15 1603 5.002 0.3%
16 1708 9.3 0.5%
17 1814 6 0.3%
18 1922 9.279 0.4%
19 2026 7.279 0.3%
20 2146 13.06 0.6%
21 2251 18.62 0.8%
22 2356 13.03 0.5%
23 2484 11.91 0.4%
24 2568 13.03 0.5%
25 2670 11.55 0.4%
26 2786 13.38 0.4%
27 2906 10.23 0.3%
28 3019 18.42 0.6%
29 3120 13.67 0.4%
30 3201 9.062 0.2%
31 3335 15.26 0.4%
32 3436 11.19 0.3%
33 3532 12.2 0.3%
34 3648 16.17 0.4%
35 3746 10.24 0.2%
36 3858 7.515 0.1%
37 3958 10.21 0.2%
38 4053 15 0.3%
39 4162 8.155 0.1%
40 4281 9.105 0.2%
41 4379 16.88 0.3%
42 4481 2.462 0.0%
43 4580 11.89 0.2%
44 4689 13.96 0.2%
45 4819 8.141 0.1%
46 4908 10.81 0.2%
47 5033 10.64 0.2%
48 5127 13.06 0.2%
49 5241 11.25 0.2%
50 5354 15.29 0.2%
51 5470 17.78 0.3%
52 5563 7.024 0.1%
53 5655 11.05 0.1%
54 5800 15.97 0.2%
55 5895 9.273 0.1%
56 5994 3.768 0.0%
57 6111 4.315 0.0%
58 6230 8.281 0.1%
59 6303 11.98 0.1%
60 6413 9.295 0.1%
61 6524 10.56 0.1%
62 6642 7.043 0.1%
63 6732 7.064 0.1%
64 6853 8.749 0.1%
65 6970 11.9 0.1%
66 7061 16.93 0.2%
67 7178 15.95 0.2%
68 7275 12.85 0.1%
69 7385 7.818 0.1%
70 7487 12.66 0.1%
71 7596 14 0.1%
72 7716 7.499 0.0%
73 7798 9.176 0.1%
74 7938 15.1 0.1%
75 8025 18.17 0.2%
76 8111 12.5 0.1%
77 8233 6.998 0.0%
78 8343 14.82 0.1%
79 8437 17.58 0.2%
80 8557 13.34 0.1%
81 8645 6.91 0.0%
82 8741 14.81 0.1%
83 8865 12.93 0.1%
84 8965 16.67 0.1%
85 9085 14.98 0.1%
86 9216 22.59 0.2%
87 9298 9.644 0.1%
88 9415 13.1 0.1%
89 9510 17.03 0.1%
90 9609 17.32 0.1%
91 9744 16.41 0.1%
92 9824 13.98 0.1%
93 9955 17.76 0.1%
94 10070 10.08 0.1%
95 10150 19.95 0.1%
96 10250 23.13 0.2%
97 10340 18.82 0.1%
98 10460 14.31 0.1%
99 10590 16.49 0.1%
100 10670 13.53 0.1%

Quality and confidence:
param error
n 0.018

Model:
Time ~= 8.34
+ n 106.8
µs

Reads = 5 + (4 * n)
Writes = 3 + (2 * n)

Parity Benchmarking Bot and others added 5 commits March 14, 2021 15:30
…n=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
@shawntabrizi
Copy link
Member

/benchmarks runtime westend auctions

@parity-benchapp
Copy link

parity-benchapp bot commented Mar 14, 2021

Finished benchmark for branch: gav-new-slots

Benchmark: Benchmark Runtime Westend Pallet

cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Results

Pallet: "auctions", Extrinsic: "new_auction", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis

-- Extrinsic Time --

Model:
Time ~= 24.1
µs

Reads = 2
Writes = 2
Min Squares Analysis

-- Extrinsic Time --

Model:
Time ~= 24.1
µs

Reads = 2
Writes = 2
Pallet: "auctions", Extrinsic: "bid", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis

-- Extrinsic Time --

Model:
Time ~= 110.3
µs

Reads = 7
Writes = 4
Min Squares Analysis

-- Extrinsic Time --

Model:
Time ~= 110.3
µs

Reads = 7
Writes = 4
Pallet: "auctions", Extrinsic: "on_initialize", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis

-- Extrinsic Time --

Model:
Time ~= 602.9
µs

Reads = 75
Writes = 71
Min Squares Analysis

-- Extrinsic Time --

Model:
Time ~= 602.9
µs

Reads = 75
Writes = 71

Parity Benchmarking Bot and others added 6 commits March 14, 2021 16:51
…n=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
@shawntabrizi
Copy link
Member

/benchmarks runtime westend slots

@parity-benchapp
Copy link

parity-benchapp bot commented Mar 14, 2021

Finished benchmark for branch: gav-new-slots

Benchmark: Benchmark Runtime Westend Pallet

cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

Results

Pallet: "slots", Extrinsic: "force_lease", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis

-- Extrinsic Time --

Model:
Time ~= 45.64
µs

Reads = 2
Writes = 2
Min Squares Analysis

-- Extrinsic Time --

Model:
Time ~= 45.64
µs

Reads = 2
Writes = 2
Pallet: "slots", Extrinsic: "manage_lease_period_start", Lowest values: [], Highest values: [], Steps: [50], Repeat: 20
Median Slopes Analysis

-- Extrinsic Time --

Model:
Time ~= 0
+ c 17.86
+ t 33.59
µs

Reads = 4 + (1 * c) + (2 * t)
Writes = 1 + (1 * c) + (2 * t)
Min Squares Analysis

-- Extrinsic Time --

Data points distribution:
c t mean µs sigma µs %
1 100 3173 6.701 0.2%
2 100 3186 11.05 0.3%
3 100 3229 15.63 0.4%
4 100 3232 12.31 0.3%
5 100 3259 13.31 0.4%
6 100 3288 8.156 0.2%
7 100 3293 13.69 0.4%
8 100 3297 9.111 0.2%
9 100 3330 10.42 0.3%
10 100 3333 10.5 0.3%
11 100 3349 9.378 0.2%
12 100 3364 12.23 0.3%
13 100 3377 10.24 0.3%
14 100 3403 17.72 0.5%
15 100 3433 14.68 0.4%
16 100 3439 11.4 0.3%
17 100 3445 13.08 0.3%
18 100 3499 10.16 0.2%
19 100 3505 12.37 0.3%
20 100 3522 6.12 0.1%
21 100 3518 12.56 0.3%
22 100 3566 14.71 0.4%
23 100 3583 14.64 0.4%
24 100 3572 14.91 0.4%
25 100 3630 15.91 0.4%
26 100 3619 16.55 0.4%
27 100 3627 21.4 0.5%
28 100 3645 16 0.4%
29 100 3686 8.761 0.2%
30 100 3680 10.16 0.2%
31 100 3702 10.34 0.2%
32 100 3708 14.51 0.3%
33 100 3755 19.02 0.5%
34 100 3748 7.092 0.1%
35 100 3759 11.05 0.2%
36 100 3773 13.07 0.3%
37 100 3830 12.67 0.3%
38 100 3832 13.97 0.3%
39 100 3857 12.14 0.3%
40 100 3853 13.77 0.3%
41 100 3870 9.754 0.2%
42 100 3922 14.68 0.3%
43 100 3937 13.1 0.3%
44 100 3950 13.35 0.3%
45 100 3957 14.03 0.3%
46 100 3966 11.32 0.2%
47 100 4010 13.58 0.3%
48 100 4017 13.86 0.3%
49 100 4026 11.66 0.2%
50 100 4017 13.66 0.3%
51 100 4040 15.45 0.3%
52 100 4072 15.01 0.3%
53 100 4102 18.7 0.4%
54 100 4127 12.61 0.3%
55 100 4131 15.12 0.3%
56 100 4135 12.18 0.2%
57 100 4161 17.57 0.4%
58 100 4192 14.51 0.3%
59 100 4186 9.229 0.2%
60 100 4231 16.1 0.3%
61 100 4234 14.75 0.3%
62 100 4241 14.73 0.3%
63 100 4279 12.66 0.2%
64 100 4282 16.46 0.3%
65 100 4324 12.37 0.2%
66 100 4303 12.64 0.2%
67 100 4368 17.88 0.4%
68 100 4361 8.294 0.1%
69 100 4367 8.823 0.2%
70 100 4358 16.84 0.3%
71 100 4432 13.56 0.3%
72 100 4436 16.17 0.3%
73 100 4485 14.86 0.3%
74 100 4503 10.35 0.2%
75 100 4480 10.73 0.2%
76 100 4487 9.452 0.2%
77 100 4562 12.6 0.2%
78 100 4530 17.02 0.3%
79 100 4549 15.78 0.3%
80 100 4617 12.28 0.2%
81 100 4607 8.46 0.1%
82 100 4631 11.47 0.2%
83 100 4625 9.829 0.2%
84 100 4666 16.63 0.3%
85 100 4676 9.005 0.1%
86 100 4681 14.24 0.3%
87 100 4723 14.82 0.3%
88 100 4736 11.32 0.2%
89 100 4771 16.05 0.3%
90 100 4748 9.846 0.2%
91 100 4785 19.73 0.4%
92 100 4842 10.97 0.2%
93 100 4811 12.98 0.2%
94 100 4835 10.63 0.2%
95 100 4859 14.45 0.2%
96 100 4877 11.62 0.2%
97 100 4922 9.871 0.2%
98 100 4955 19.75 0.3%
99 100 4927 15.72 0.3%
100 1 1625 13.97 0.8%
100 2 1655 8.693 0.5%
100 3 1695 9.59 0.5%
100 4 1726 12.17 0.7%
100 5 1745 8.774 0.5%
100 6 1785 12.51 0.7%
100 7 1807 6.677 0.3%
100 8 1862 14.56 0.7%
100 9 1893 8.039 0.4%
100 10 1928 8.792 0.4%
100 11 1959 10.16 0.5%
100 12 1977 9.392 0.4%
100 13 2019 7.745 0.3%
100 14 2049 8.428 0.4%
100 15 2077 4.978 0.2%
100 16 2095 10.52 0.5%
100 17 2148 11.43 0.5%
100 18 2176 2.921 0.1%
100 19 2201 11.38 0.5%
100 20 2232 11.87 0.5%
100 21 2276 7.807 0.3%
100 22 2306 15.85 0.6%
100 23 2353 13.74 0.5%
100 24 2367 10.13 0.4%
100 25 2403 13.38 0.5%
100 26 2418 9.282 0.3%
100 27 2476 18.04 0.7%
100 28 2489 9.308 0.3%
100 29 2514 5.216 0.2%
100 30 2575 14.41 0.5%
100 31 2605 6.756 0.2%
100 32 2623 6.428 0.2%
100 33 2651 18.26 0.6%
100 34 2714 11.92 0.4%
100 35 2744 16.1 0.5%
100 36 2759 13.08 0.4%
100 37 2799 13.49 0.4%
100 38 2815 13.9 0.4%
100 39 2850 7.037 0.2%
100 40 2881 10.63 0.3%
100 41 2921 18.49 0.6%
100 42 2942 11.38 0.3%
100 43 2979 8.283 0.2%
100 44 3041 8.405 0.2%
100 45 3057 6.853 0.2%
100 46 3082 9.684 0.3%
100 47 3129 10.91 0.3%
100 48 3159 14.87 0.4%
100 49 3183 16.49 0.5%
100 50 3225 9.751 0.3%
100 51 3244 8.74 0.2%
100 52 3287 15.28 0.4%
100 53 3335 14.7 0.4%
100 54 3341 8.67 0.2%
100 55 3414 13.38 0.3%
100 56 3449 8.311 0.2%
100 57 3468 12.12 0.3%
100 58 3517 9.284 0.2%
100 59 3551 11.95 0.3%
100 60 3586 9.062 0.2%
100 61 3598 8.81 0.2%
100 62 3627 6.935 0.1%
100 63 3673 7.045 0.1%
100 64 3700 11.29 0.3%
100 65 3745 13.69 0.3%
100 66 3785 12.41 0.3%
100 67 3805 14.32 0.3%
100 68 3853 14.48 0.3%
100 69 3880 14.39 0.3%
100 70 3917 14.89 0.3%
100 71 3933 10.33 0.2%
100 72 3989 6.239 0.1%
100 73 4009 7.419 0.1%
100 74 4056 16.96 0.4%
100 75 4072 18.61 0.4%
100 76 4114 10.81 0.2%
100 77 4144 11.67 0.2%
100 78 4173 9.766 0.2%
100 79 4230 9.589 0.2%
100 80 4274 6.819 0.1%
100 81 4297 10.47 0.2%
100 82 4352 14.16 0.3%
100 83 4365 12.23 0.2%
100 84 4402 8.128 0.1%
100 85 4439 13.16 0.2%
100 86 4481 16.58 0.3%
100 87 4484 15.56 0.3%
100 88 4532 8.818 0.1%
100 89 4608 12.84 0.2%
100 90 4609 9.385 0.2%
100 91 4645 8.845 0.1%
100 92 4673 13.97 0.2%
100 93 4711 19.63 0.4%
100 94 4730 11.33 0.2%
100 95 4773 8.3 0.1%
100 96 4815 27.47 0.5%
100 97 4825 15.73 0.3%
100 98 4879 20.11 0.4%
100 99 4937 12.73 0.2%
100 100 4954 17.29 0.3%

Quality and confidence:
param error
c 0.019
t 0.019

Model:
Time ~= 0
+ c 17.79
+ t 33.77
µs

Reads = 4 + (1 * c) + (2 * t)
Writes = 1 + (1 * c) + (2 * t)

Copy link
Member

@shawntabrizi shawntabrizi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking to merge these unaudited changes so that we and other users can start to test things locally.

@shawntabrizi shawntabrizi merged commit 0f00a78 into master Mar 14, 2021
@shawntabrizi shawntabrizi deleted the gav-new-slots branch March 14, 2021 21:39
@shawntabrizi shawntabrizi mentioned this pull request Mar 14, 2021
5 tasks
ordian pushed a commit that referenced this pull request Mar 19, 2021
* TODOs

* Add auctions.rs, comment on changes needed.

* Remove cruft from slots

* Remove more from auctions.rs

* More logic drafting in slots.

* More logic in slots.rs

* patch some errors

* more fixes

* last nit

* Cleanups in slots.rs

* Cleanups in slots.rs

* patches

* make build

* crowdloan to new api

* auction compile

* Use ParaId instead of FundIndex in Crowdloan (#2303)

* use paraid instead of fundindex

* Update crowdloan.rs

* check caller is manager

* Auction tests and fix build warnings.

* Configurable origin for initiating auctions

* Remove on_finalize

* #2303 (manual merge)

* Tests for Slots

* some registrar tests

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update runtime/common/src/slots.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Slots uses Registrar for CurrentChains

* swap works test

* on swap impl

* traitify parachain cleanup

* explicit lifecycle tracking for paras

* initial implementation of lifecycles and upgrades

* clean up a bit

* Update runtime/common/src/slots.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* fix doc comment

* more rigid lifecycle checks

* include paras which are transitioning, and lifecycle query

* format guide

* update api

* update guide

* explicit outgoing state, fix genesis

* handle outgoing with transitioning paras

* Revert "explicit lifecycle tracking for paras"

This reverts commit 4177af7.

* remove lifecycle tracking from registrar

* do not include transitioning paras in identifier

* Update paras_registrar.rs

* final patches to registrar

* Fix test

* use noop in test

* clean up pending swap on deregistration

* finish registrar tests

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Apply suggestions from code review

* Use matches macro

* Correct terms

* Apply suggestions from code review

* Remove direct need for Slots and Registrar from Crowdloan

* Rejig things slightly

* actions queue

* Revert "actions queue"

This reverts commit b2e9011.

* Traitify Auction interface.

* Mockups and initial code for Crowdloan testing

* One test...

* collapse onboarding state

* fix some crowdloan tests

* one more

* start benchmarks for auctions

* benchmark bid

* fix more crowdloan tests

* onboard and begin retirement no longer exist

* Revert "onboard and begin retirement no longer exist"

This reverts commit 2e100fd.

* Simplify crowdloan and make it work.

* Fixes

* fix some

* finish merge fixes

* fix refund bug in auctions

* Add traits to Registrar for tests and benchmarks

* fix more auction benchmarks

* Fix TestAuctioneer

* finish crowdloan benchmarks

* start setting up full integration tests

* expand integration tests

* finish basic integration test

* add more integration tests

* begin slots benchmarks

* start paras registrar benchmarks

* fix merge

* fix tests

* clean up paras registrar

* remove println

* remove outdated cleanup config

* update benchmarks

* Add WeightInfo

* enable runtime-benchmarks feature flag

* complete swap benchmark

* add parachains and onboarding into westend

* add benchmarks and genesis

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* fix benchmark execution

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* Use `new_raise_len` in crowdloan on_initialize

* Update paras_registrar.rs

* fix westend merge

* impl on_swap for crowdloan

* Check fund exists before create

* update for crowdloan sig

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* slots on_initialize

* use integration tests environment for benchmarks

* fix hrmp event

* auction on_initialize

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* fix storage name in auctions

* add auction_index to winning data

* winning data takes into account current auction index

* remove println

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* Revert "add auction_index to winning data"

* PastRandomness.

* Fixes

* Use new randomness

* fix use of randomness in auctions and runtime config

* expose consts

* fix auction test

* add deposit per byte for para registration

* basic swap integration test

* make swap test more comprehensive

* Add WinningVec for easier retrieval in the front-end.

* clean up `WinningVec` at the end

* Add event for when a new best bid comes in

* Fix propagation of winners in ending period

* fix benchmarks, refund weight in dissolve

* fix unused

* remove some TODOs

* setup opaque keys for paras in westend

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* remove unused

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* back to regular runtime config

* use saturating math where user input can be

* better first slot check

* Update runtime/common/src/claims.rs

* update westend onswap impl

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
ordian added a commit that referenced this pull request Mar 19, 2021
* master:
  Don't accept incoming connections for collators (#2644)
  Improve the logging (#2645)
  Update for the new substrate client API (#2570)
  integrate faster erasure code (#2608)
  Companion for #8372 (Replace 'Module' with 'Pallet' in construct_runtime macro) (#2629)
  Request based collation fetching (#2621)
  Companion for Substrate#8386 (#2634)
  Polkadot companion for Substrate PR #7640 (Store multiple Justifications per block) (#2358)
  yet another set of logging improvements (#2638)
  Reduce number of active leaves at startup (#2631)
  re benchmark  (#2630)
  Fix wrong deposit amount in council voters. (#2562)
  Add /data symlink to Docker containers (#2627)
  Companion for sub/8176 (#2622)
  Remove TODO from substrate#2986 (#2628)
  update ring to 0.16.20 (#2626)
  New slots/auctions architecture (#2294)
  add tracing when no assignment in candidate selection (#2623)
  Backing and collator protocol traces including para-id (#2620)
  more diagnostic logs for approval-voting (#2618)
ordian added a commit that referenced this pull request Mar 24, 2021
* ci: initial fuzzer job

* erasure-coding: update fuzzer Cargo.lock

* syntax fix

* try this first

* install honggfuzz deps

* try not

* try if else

* try SIGINT

* ignore hfuzz dirs

* ???

* bash is growing on me

* decouple builds from running

* fix a typo

* try copying dirs

* fix indentation

* try using absolute paths

* another try

* caching is not worth it

* remove outdated needs

* cleanup and add futher TODOs

* Update .github/workflows/honggfuzz.yml

* more diagnostic logs for approval-voting (#2618)

* Backing and collator protocol traces including para-id (#2620)

* improve backing/provisioner spans

* span for collation requests

* add para_id to unbacked candidate spans

* differentiate validation-construction and find-assignment in selection

* better find-assignment spans

* organize unbacked-candidate spans directly under job root

* Update node/core/provisioner/src/lib.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Andronik Ordian <write@reusable.software>

* add tracing when no assignment in candidate selection (#2623)

* New slots/auctions architecture (#2294)

* TODOs

* Add auctions.rs, comment on changes needed.

* Remove cruft from slots

* Remove more from auctions.rs

* More logic drafting in slots.

* More logic in slots.rs

* patch some errors

* more fixes

* last nit

* Cleanups in slots.rs

* Cleanups in slots.rs

* patches

* make build

* crowdloan to new api

* auction compile

* Use ParaId instead of FundIndex in Crowdloan (#2303)

* use paraid instead of fundindex

* Update crowdloan.rs

* check caller is manager

* Auction tests and fix build warnings.

* Configurable origin for initiating auctions

* Remove on_finalize

* #2303 (manual merge)

* Tests for Slots

* some registrar tests

* Apply suggestions from code review

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update runtime/common/src/slots.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Slots uses Registrar for CurrentChains

* swap works test

* on swap impl

* traitify parachain cleanup

* explicit lifecycle tracking for paras

* initial implementation of lifecycles and upgrades

* clean up a bit

* Update runtime/common/src/slots.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* fix doc comment

* more rigid lifecycle checks

* include paras which are transitioning, and lifecycle query

* format guide

* update api

* update guide

* explicit outgoing state, fix genesis

* handle outgoing with transitioning paras

* Revert "explicit lifecycle tracking for paras"

This reverts commit 4177af7.

* remove lifecycle tracking from registrar

* do not include transitioning paras in identifier

* Update paras_registrar.rs

* final patches to registrar

* Fix test

* use noop in test

* clean up pending swap on deregistration

* finish registrar tests

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Apply suggestions from code review

* Use matches macro

* Correct terms

* Apply suggestions from code review

* Remove direct need for Slots and Registrar from Crowdloan

* Rejig things slightly

* actions queue

* Revert "actions queue"

This reverts commit b2e9011.

* Traitify Auction interface.

* Mockups and initial code for Crowdloan testing

* One test...

* collapse onboarding state

* fix some crowdloan tests

* one more

* start benchmarks for auctions

* benchmark bid

* fix more crowdloan tests

* onboard and begin retirement no longer exist

* Revert "onboard and begin retirement no longer exist"

This reverts commit 2e100fd.

* Simplify crowdloan and make it work.

* Fixes

* fix some

* finish merge fixes

* fix refund bug in auctions

* Add traits to Registrar for tests and benchmarks

* fix more auction benchmarks

* Fix TestAuctioneer

* finish crowdloan benchmarks

* start setting up full integration tests

* expand integration tests

* finish basic integration test

* add more integration tests

* begin slots benchmarks

* start paras registrar benchmarks

* fix merge

* fix tests

* clean up paras registrar

* remove println

* remove outdated cleanup config

* update benchmarks

* Add WeightInfo

* enable runtime-benchmarks feature flag

* complete swap benchmark

* add parachains and onboarding into westend

* add benchmarks and genesis

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* fix benchmark execution

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* Use `new_raise_len` in crowdloan on_initialize

* Update paras_registrar.rs

* fix westend merge

* impl on_swap for crowdloan

* Check fund exists before create

* update for crowdloan sig

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* slots on_initialize

* use integration tests environment for benchmarks

* fix hrmp event

* auction on_initialize

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* fix storage name in auctions

* add auction_index to winning data

* winning data takes into account current auction index

* remove println

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* Revert "add auction_index to winning data"

* PastRandomness.

* Fixes

* Use new randomness

* fix use of randomness in auctions and runtime config

* expose consts

* fix auction test

* add deposit per byte for para registration

* basic swap integration test

* make swap test more comprehensive

* Add WinningVec for easier retrieval in the front-end.

* clean up `WinningVec` at the end

* Add event for when a new best bid comes in

* Fix propagation of winners in ending period

* fix benchmarks, refund weight in dissolve

* fix unused

* remove some TODOs

* setup opaque keys for paras in westend

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* remove unused

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* back to regular runtime config

* use saturating math where user input can be

* better first slot check

* Update runtime/common/src/claims.rs

* update westend onswap impl

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>

* update ring to 0.16.20 (#2626)

* Remove TODO from substrate#2986 (#2628)

* Companion for sub/8176 (#2622)

* Merge

* Fixes

* Fix build

* remove dep.

* undo dep.

* upadte substrate

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* Fix lock

* revert lock; cargo update -p sp-io

* from_rational_approx -> from_rational

* Silence more warnings

Co-authored-by: Gav Wood <gavin@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>

* Add /data symlink to Docker containers (#2627)

* add /data symlink to Docker

* fix comments

* Fix wrong deposit amount in council voters. (#2562)

* Fix wrong deposit amount in council voters.

* Fix some build

* make it all compile.. so far.

* Fix

* break build

* Okay fix it again

* re benchmark  (#2630)

* Change something

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_staking --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

* cargo run --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_election_provider_multi_phase --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/

Co-authored-by: Parity Benchmarking Bot <admin@parity.io>

* Reduce number of active leaves at startup (#2631)

Currently we will take all leaves and give that to the overseer on
startup, but this is a bad idea when the finality is lagging for
example. There can be many of unfinalized leaves, we don't even need to
look at anymore. To solve this, the pr adds a maximum of 4 leaves we
forward to the overseer and the pr also checks that we only pass uncles
of the best block.

* yet another set of logging improvements (#2638)

* Polkadot companion for Substrate PR #7640 (Store multiple Justifications per block) (#2358)

* service: update for substrate PR #7640

* update substrate

* Add Pallet Babe to Integration Tests Runtime

Co-authored-by: André Silva <andrerfosilva@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Companion for Substrate#8386 (#2634)

* Companion for Substrate#8386

paritytech/substrate#8386

* "Update Substrate"

Co-authored-by: parity-processbot <>

* Request based collation fetching (#2621)

* Introduce collation fetching protocol

also move to mod.rs

* Allow `PeerId`s in requests to network bridge.

* Fix availability distribution tests.

* Move CompressedPoV to primitives.

* Request based collator protocol: validator side

- Missing: tests
- Collator side
- don't connect, if not connected

* Fixes.

* Basic request based collator side.

* Minor fix on collator side.

* Don't connect in requests in collation protocol.

Also some cleanup.

* Fix PoV distribution

* Bump substrate

* Add back metrics + whitespace fixes.

* Add back missing spans.

* More cleanup.

* Guide update.

* Fix tests

* Handle results in tests.

* Fix weird compilation issue.

* Add missing )

* Get rid of dead code.

* Get rid of redundant import.

* Fix runtime build.

* Cleanup.

* Fix wasm build.

* Format fixes.

Thanks @andronik !

* Companion for #8372 (Replace 'Module' with 'Pallet' in construct_runtime macro) (#2629)

* Replace 'Module' with 'Pallet'.

* "Update Substrate"

* fix babe usage

* fix benchmark

Co-authored-by: parity-processbot <>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>

* integrate faster erasure code (#2608)

Breaks compatibility for distributing PoV and PersistentValidationData between validators.

Ref #2442

* Update for the new substrate client API (#2570)

* Update for the new substrate client API

* Code review suggestions

* Update substrate

* Improve the logging (#2645)

* Don't accept incoming connections for collators (#2644)

* Don't accept incoming connections for collators

on the `Collation` peer set.

* Better docs.

* fix reconstruct fuzzer name

* make script more robust

* REVERTME: test run

* REVERTME: test run II

* Revert "REVERTME: test run II"

This reverts commit 58375df.

* Revert "REVERTME: test run"

This reverts commit 9759cb6.

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Martin Pugh <pugh@s3kr.it>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Jon Häggblad <jon.haggblad@gmail.com>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com>
Co-authored-by: Shaun Wang <spxwang@gmail.com>
Co-authored-by: Bernhard Schuster <bernhard@ahoi.io>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. C1-low PR touches the given topic and has a low impact on builders.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants