Coretime auto-renew #4424

Szegoo · 2024-05-09T16:56:41Z

This PR adds functionality that allows tasks to enable auto-renewal. Each task eligible for renewal can enable auto-renewal.

A new storage value is added to track all the cores with auto-renewal enabled and the associated task running on the core. The BoundedVec is sorted by CoreIndex to make disabling auto-renewal more efficient.

Cores are renewed at the start of a new bulk sale. If auto-renewal fails(e.g. due to the sovereign account of the task not holding sufficient balance), an event will be emitted, and the renewal will continue for the other cores.

The two added extrinsics are:

enable_auto_renew: Extrinsic for enabling auto renewal.
disable_auto_renew: Extrinsic for disabling auto renewal.

TODOs:

Write benchmarks for the newly added extrinsics.

Closes: #4351

eskimor

If it is the sovereign account of the para, why don't we implement auto renew on the para itself? Major reason, I guess: It would be impossible to mess up the timing. Otherwise each para would need to keep track of the sale cycle on Coretime.

substrate/frame/broker/src/lib.rs

eskimor · 2024-05-10T12:00:27Z

substrate/frame/broker/src/lib.rs

@@ -172,6 +181,11 @@ pub mod pallet {
 	#[pallet::storage]
 	pub type CoreCountInbox<T> = StorageValue<_, CoreIndex, OptionQuery>;

+	/// Keeping track of cores which have auto-renewal enabled.
+	#[pallet::storage]


You have an invariant that this list is supposed to be sorted. This should the very least be documented here.

Yes, keep in mind that this PR is still very much WIP.

PR is merged without this being really addressed, should be a 1 liner fn try_state

Szegoo · 2024-05-10T12:26:37Z

If it is the sovereign account of the para, why don't we implement auto renew on the para itself?

But what if the parachain has multiple cores assigned to it? For example, perhaps the parachain wants to keep renewing only one core but has acquired an additional core for this bulk period due to higher demand.

The reason I am storing the TaskId here as well is to avoid having to look it up for each core when performing auto-renewals.

Szegoo · 2024-05-17T08:27:00Z

@eskimor I would appreciate a review to see if what I implemented here is sensible before writing benchmarks.

Polkadot-Forum · 2024-05-21T17:08:01Z

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/kusama-coretime-renewal/8219/1

substrate/frame/broker/src/tick_impls.rs

substrate/frame/broker/src/dispatchable_impls.rs

substrate/frame/broker/src/benchmarking.rs

paritytech-cicd-pr · 2024-07-01T09:15:09Z

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: cargo-clippy
Logs: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6592890

franciscoaguirre

Looks good to me. @seadanda ?

seadanda

Nice, just did the last pass over your changes since my last review. Thanks!

kianenigma · 2024-08-05T15:14:15Z

substrate/frame/broker/src/lib.rs

+	/// Sorted by `CoreIndex` to make the removal of cores from auto-renewal more efficient.
+	#[pallet::storage]
+	pub type AutoRenewals<T: Config> =
+		StorageValue<_, BoundedVec<AutoRenewalRecord, T::MaxAutoRenewals>, ValueQuery>;


Sorry for being a doomer, but isn't this super not scalable? As in, up to N renewals can happen at most, and all of it is on_init, in a single block?

Moreover, if there is congestion (e.g. more than MaxAutoRenewals cores that wanna auto-renew), how do we deal with congestion? The right way to solve this kind of issue is to let user compete via more tx-fees.

I suppose this is still assuming we will only have a maximum of a few dozen cores that want to auto-renew, which is fine, but I think we could have done a bit better.

You are right; there is an assumption that there is a fairly limited number of cores and also a limited amount that will own an entire core and do auto-renewals. However, what you mentioned is valid, but I don't think this will be an issue until the number of available cores drastically increases.

I am not really sure what you meant by 'let users compete via more tx-fees.'

* master: (51 commits) Remove unused feature gated code from the minimal template (#5237) make polkadot-parachain startup errors pretty (#5214) Coretime auto-renew (#4424) network/strategy: Backoff and ban overloaded peers to avoid submitting the same request multiple times (#5029) Fix frame crate usage doc (#5222) beefy: Tolerate pruned state on runtime API call (#5197) rpc: Enable ChainSpec for polkadot-parachain (#5205) Add an adapter for configuring AssetExchanger (#5130) Replace env_logger with sp_tracing (#5065) Adjust sync templates flow to use new release branch (#5182) litep2p/discovery: Publish authority records with external addresses only (#5176) Run UI tests in CI for some other crates (#5167) Remove `pallet::getter` usage from the pallet-balances (#4967) pallet-timestamp: `UnixTime::now` implementation logs error only if called at genesis (#5055) [CI] Cache try-runtime check (#5179) [Backport] version bumps and the prdocs reordering from stable2407 (#5178) [subsystem-benchmark] Update availability-distribution-regression-bench baseline after recent subsystem changes (#5180) Remove pallet::getter usage from proxy (#4963) Remove pallet::getter macro usage from pallet-election-provider-multi-phase (#4487) Review-bot@2.6.0 (#5177) ...

This PR adds functionality that allows tasks to enable auto-renewal. Each task eligible for renewal can enable auto-renewal. A new storage value is added to track all the cores with auto-renewal enabled and the associated task running on the core. The `BoundedVec` is sorted by `CoreIndex` to make disabling auto-renewal more efficient. Cores are renewed at the start of a new bulk sale. If auto-renewal fails(e.g. due to the sovereign account of the task not holding sufficient balance), an event will be emitted, and the renewal will continue for the other cores. The two added extrinsics are: - `enable_auto_renew`: Extrinsic for enabling auto renewal. - `disable_auto_renew`: Extrinsic for disabling auto renewal. TODOs: - [x] Write benchmarks for the newly added extrinsics. Closes: paritytech#4351 --------- Co-authored-by: Dónal Murray <donalm@seadanda.dev>

Polkadot-Forum · 2024-10-21T09:00:56Z

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/polkadot-coretime-renewals/10536/1

Szegoo added 4 commits May 4, 2024 09:02

start

afb5e75

setup

2716e0f

outlining the main logic

e443c9e

bounded vec

2a8a68b

eskimor reviewed May 10, 2024

View reviewed changes

Szegoo added 15 commits May 11, 2024 09:19

resolvinig some todos

8df2f81

auto-renew for legacy leases

82859ad

init test

e2a4dd5

more tests

9c1c7e3

enable_auto_renew_renews

c7ccf5a

different approach

c3eaa21

docs

edffbfc

enable_auto_renewal_with_end_hint_works

68f244c

auto_renewal_works

893fa9d

progress

0b50a58

store account instead of task

9dc9b6a

fix builds

9baa77a

disable auto-renewal

ee7e3c4

disable auto renew test

e0f1c71

docs

1a46abc

Szegoo marked this pull request as ready for review May 17, 2024 08:25

Szegoo requested a review from a team as a code owner May 17, 2024 08:25

seadanda reviewed May 22, 2024

View reviewed changes

substrate/frame/broker/src/tick_impls.rs Outdated Show resolved Hide resolved

account for changing core indices

2f17544

paritytech-review-bot bot requested a review from a team May 23, 2024 09:15

Szegoo added 2 commits May 23, 2024 11:57

fix

63b6f66

bench enable_auto_renew

aaba50d

seadanda reviewed Jun 25, 2024

View reviewed changes

substrate/frame/broker/src/dispatchable_impls.rs Show resolved Hide resolved

seadanda reviewed Jun 25, 2024

View reviewed changes

substrate/frame/broker/src/dispatchable_impls.rs Show resolved Hide resolved

substrate/frame/broker/src/benchmarking.rs Show resolved Hide resolved

fix benchmarks

ef9c4fb

paritytech-review-bot bot requested a review from a team July 1, 2024 08:53

Szegoo and others added 2 commits July 1, 2024 11:02

Merge branch 'master' into auto-renew

4d35d2b

merge fixes

15d9e1b

Szegoo and others added 6 commits July 1, 2024 11:38

benchmark fix

eff4ee7

resolve conflicts

61f466d

prdoc

d2041f1

improve docs

7155233

improve comments

9a90fbd

Merge branch 'master' into auto-renew

0cf0dad

Szegoo requested a review from seadanda August 1, 2024 16:15

Merge branch 'master' into auto-renew

c1fc125

franciscoaguirre approved these changes Aug 5, 2024

View reviewed changes

seadanda approved these changes Aug 5, 2024

View reviewed changes

seadanda added this pull request to the merge queue Aug 5, 2024

seadanda self-assigned this Aug 5, 2024

Merged via the queue into paritytech:master with commit f170af6 Aug 5, 2024
162 of 163 checks passed

kianenigma reviewed Aug 5, 2024

View reviewed changes

ggwpez mentioned this pull request Sep 9, 2024

Parent issue for stable2409 LTS release #5583

Closed

github-actions bot mentioned this pull request Oct 8, 2024

Update polkadot-sdk from stable2407 to stable2409 moonbeam-foundation/moonbeam#2994

Open

TDemeco mentioned this pull request Oct 14, 2024

feat: ⏫ upgrade to Polkadot SDK stable2409 Moonsong-Labs/storage-hub#228

Merged

gui1117 mentioned this pull request Nov 14, 2024

pallet broker: some benchmarks are weightless for kitchensink runtime. Resulting in substrate weight being 0 #6474

Open

seadanda mentioned this pull request Nov 15, 2024

[pallet-broker] Fix auto renew benchmarks #6505

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Coretime auto-renew #4424

Coretime auto-renew #4424

Szegoo commented May 9, 2024 •

edited

Loading

eskimor left a comment •

edited

Loading

eskimor May 10, 2024

Szegoo May 10, 2024

kianenigma Aug 5, 2024

Szegoo commented May 10, 2024 •

edited

Loading

Szegoo commented May 17, 2024

Polkadot-Forum commented May 21, 2024

paritytech-cicd-pr commented Jul 1, 2024

franciscoaguirre left a comment

seadanda left a comment

kianenigma Aug 5, 2024

Szegoo Aug 5, 2024 •

edited

Loading

Polkadot-Forum commented Oct 21, 2024

Coretime auto-renew #4424

Coretime auto-renew #4424

Conversation

Szegoo commented May 9, 2024 • edited Loading

eskimor left a comment • edited Loading

Choose a reason for hiding this comment

eskimor May 10, 2024

Choose a reason for hiding this comment

Szegoo May 10, 2024

Choose a reason for hiding this comment

kianenigma Aug 5, 2024

Choose a reason for hiding this comment

Szegoo commented May 10, 2024 • edited Loading

Szegoo commented May 17, 2024

Polkadot-Forum commented May 21, 2024

paritytech-cicd-pr commented Jul 1, 2024

franciscoaguirre left a comment

Choose a reason for hiding this comment

seadanda left a comment

Choose a reason for hiding this comment

kianenigma Aug 5, 2024

Choose a reason for hiding this comment

Szegoo Aug 5, 2024 • edited Loading

Choose a reason for hiding this comment

Polkadot-Forum commented Oct 21, 2024

Szegoo commented May 9, 2024 •

edited

Loading

eskimor left a comment •

edited

Loading

Szegoo commented May 10, 2024 •

edited

Loading

Szegoo Aug 5, 2024 •

edited

Loading