Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-Block-Migrations: Support small steps #4989

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
Open

Conversation

ggwpez
Copy link
Member

@ggwpez ggwpez commented Jul 9, 2024

Multi-Block-Migrations currently require the migrations to do as much compute per step as possible.
This was done to allow for more efficient batch code to run, but from a user perspective it can be much nicer to be able to do just a tiny step (eg migrating a map key).

This MR changes MBM to be able to do mini steps, but stays backwards compatible to MBMs that do big steps.

Changes:

  • Add max-steps limit to MBMs
  • Invoke MBMs multiple times per block until all weight is used up

Integration

You can now optionally implement this function for your MBMs:

impl SteppedMigration for MyMigration {
	...

+	fn max_steps() -> Option<u32> {
+	 	None
+	}
}

Not implementing it is defaulting to no limit - same as before.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
@ggwpez ggwpez added the T1-FRAME This PR/Issue is related to core FRAME, the framework. label Jul 22, 2024
ggwpez added 11 commits July 22, 2024 17:26
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
@ggwpez ggwpez marked this pull request as ready for review July 24, 2024 19:42
@ggwpez ggwpez requested a review from a team as a code owner July 24, 2024 19:42
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Comment on lines 755 to 759
Self::deposit_event(Event::MigrationAdvanced {
index: cursor.index,
took_blocks,
took_steps,
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we keep that event? I am just thinking that it's going to be a lot of deposited event (1 for each iteration)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes i also wondered it. Maybe only at the end of the loop... will try how it looks.

Emitting one event per iteration could be problematic when a
migration does thousands of steps per block, and hardly that
useful. We therefore only emit one MigrationAdvanced event at the
end of the block.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
@paritytech-cicd-pr
Copy link

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: cargo-clippy
Logs: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6846782

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T1-FRAME This PR/Issue is related to core FRAME, the framework.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants