Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🔷 [Project Tracking] Resharding v3 #11881

Open
28 of 70 tasks
wacban opened this issue Aug 5, 2024 · 2 comments
Open
28 of 70 tasks

🔷 [Project Tracking] Resharding v3 #11881

wacban opened this issue Aug 5, 2024 · 2 comments
Assignees
Labels
Near Core T-core Team: issues relevant to the core team

Comments

@wacban
Copy link
Contributor

wacban commented Aug 5, 2024

Goals

  • The ability to increase the blockchain capacity by increasing the number of shards
  • The ability to balance the load across shards
  • The ability to do the above fast
  • The ability to do the above automatically (dynamic resharding)

References

Resharding Docs
Resharding V2 tracking issue

Tasks

Design - Instant Resharding

Design - Continuous Balancing @Longarithm & @Trisfald

  • High level design
  • Design document WIP
    • doc
    • How to handle large accounts?
      • account transfer
      • balancing manager should prevent large / busy accounts from hopping back and forth
    • Estimate and address the balancing overhead.
    • Account availability during transfer.
    • How to handle all sorts of receipts?
    • Estimate impact on the archival node storage.
    • Consider implementing a limit on account size.
    • Polish and verify design assumptions

Design - Resharding v2.1 @wacban

The old resharding is not particularly interesting because it takes at least an epoch to take effect. It's here mostly for completeness but it may be also worth considering it as a milestone towards the v3. Viability depends on the overlap with the end goal design and the amount of throw away work.

  • High level design, Short Epochs
  • Suitability as milestone towards Instant Resharding

Design - Automatic Scheduling aka Dynamic Resharding

Design - Mid-Epoch / Next-Epoch Resharding

The typical resharding operations, splitting and merging shards, require changes to the validator assignment. In order to execute resharding fast we will need to enhance the assignment to be able to be adjusted without the usual two epoch delay.

  • High level design
    • doc
    • Some ideas were presented in the Instant Resharding doc
    • Mid-Epoch may not work well with State Sync :(
@wacban wacban self-assigned this Aug 5, 2024
@wacban wacban added T-core Team: issues relevant to the core team Near Core labels Aug 5, 2024
@wacban wacban changed the title 🔷 [Tracking issue] Resharding v3 🔷 [Project Tracking] Resharding v3 Aug 6, 2024
@wacban
Copy link
Contributor Author

wacban commented Sep 2, 2024

The design phase is now completed. We will be implementing the Instant Resharding approach. I will rearrange the issue to reflect the current work and I'll move the design task list down the description.

@walnut-the-cat
Copy link
Contributor

walnut-the-cat commented Sep 16, 2024

Sept 2-13

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Near Core T-core Team: issues relevant to the core team
Projects
None yet
Development

No branches or pull requests

2 participants