Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chain/migrations refactor: optionally set max migration workers #9784

Merged
merged 2 commits into from
Dec 5, 2022

Conversation

frrist
Copy link
Member

@frrist frrist commented Dec 2, 2022

Proposed Changes

Allow the maximum number of workers for state migrations to be specified via an environment variable LOTUS_MIGRATION_MAX_WORKER_COUNT

Additional Info

Limits the number of actors being migrated simultaneously, thus reducing the overall memory footprint required by migrations. Motivated by experience in production where nodes with high CPU count exhaust all available memory due to a high number of workers loading actor states into memory to perform migration.

Checklist

Before you mark the PR ready for review, please make sure that:

  • Commits have a clear commit message.
  • PR title is in the form of of <PR type>: <area>: <change being made>
    • example: fix: mempool: Introduce a cache for valid signatures
    • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
    • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
  • New features have usage guidelines and / or documentation updates in
  • Tests exist for new functionality or change in behavior
  • CI is green

- configure via LOTUS_MIGRATION_MAX_WORKER_COUNT
@frrist frrist requested a review from a team as a code owner December 2, 2022 19:42
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
@frrist frrist self-assigned this Dec 3, 2022
@magik6k magik6k merged commit 3155b34 into master Dec 5, 2022
@magik6k magik6k deleted the frrist/config-migration-worker-limit branch December 5, 2022 09:05
@travisperson travisperson mentioned this pull request Feb 14, 2023
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants