Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop head tracker for summaries DAG #6744

Open
wants to merge 1 commit into
base: unstable
Choose a base branch
from
Open

Conversation

dapplion
Copy link
Collaborator

@dapplion dapplion commented Dec 28, 2024

Issue Addressed

The head tracker is a persisted piece of state that must be kept in sync with the fork-choice. It has been a source of pruning issues in the past, so we want to remove it

When implementing tree-states in the hot DB we have to change the pruning routine (more details below) so we want to do those changes first in isolation.

Closes #1785

Proposed Changes

Current DB migration routine

  • Locate abandoned heads with head tracker
  • Use a roots iterator to collect the ancestors of those heads can be pruned
  • Delete those abandoned blocks / states
  • Migrate the newly finalized chain to the freezer

In summary, it computes what it has to delete and keeps the rest. Then it migrates data to the freezer. If the abandoned forks routine has a bug it can break the freezer migration.

Proposed migration routine (this PR)

  • Migrate the newly finalized chain to the freezer
  • Load all state summaries from disk
  • From those, just knowing the head and finalized block compute two sets: (1) descendants of finalized (2) newly finalized chain
  • Iterate all summaries, if a summary does not belong to set (1) or (2), delete

This strategy is more sound as it just checks what's there in the hot DB, computes what it has to keep and deletes the rest. Because it does not rely and 3rd pieces of data we can drop the head tracker and pruning checkpoint. Since the DB migration happens first now, as long as the computation of the sets to keep is correct we won't have pruning issues.

@dapplion
Copy link
Collaborator Author

@michaelsproul If we stop updating the head_tracker in the persisted head, how can we downgrade safely? If you run an old version of Lighthouse it will read the persisted head which contains an empty head tracker.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant