Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store changes to persist data columns #6073

Merged
merged 8 commits into from
Aug 2, 2024

Conversation

jimmygchen
Copy link
Member

@jimmygchen jimmygchen commented Jul 11, 2024

Issue Addressed

Part of #6072.

This PR contains only the Store changes to persist data columns.

DataColumns are stored individually by index. Sampling peers will request single columns or arbitrary slots. Storing bundled like blobs would be inefficient.

Upstream PRs:

Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com>
@jimmygchen jimmygchen added ready-for-review The code is ready for review das Data Availability Sampling labels Jul 11, 2024
# Conflicts:
#	beacon_node/store/src/lib.rs
#	consensus/types/src/chain_spec.rs
beacon_node/store/src/lib.rs Outdated Show resolved Hide resolved
beacon_node/store/src/hot_cold_store.rs Outdated Show resolved Hide resolved
@jimmygchen jimmygchen requested a review from realbigsean August 1, 2024 07:14
Copy link
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Although this introduces new columns on disk I think it is OK to merge this without a schema version increment as nothing will be stored in those columns until the fork epoch is set.

Ideally we would want to block downgrading to a column-oblivious schema once peer DAS is activated, but it's kind of a moot point because nodes won't be able to follow the chain without these changes (and the rest of the DAS logic).

We will also be bumping the schema version (and disallowing downgrades) with this PR:

beacon_node/store/src/hot_cold_store.rs Outdated Show resolved Hide resolved
beacon_node/store/src/hot_cold_store.rs Outdated Show resolved Hide resolved
@michaelsproul
Copy link
Member

@mergify queue

Copy link

mergify bot commented Aug 2, 2024

queue

🛑 The pull request has been removed from the queue default

The pull request #6073 has been manually updated.

You can take a look at Queue: Embarked in merge queue check runs for more details.

In case of a failure due to a flaky test, you should first retrigger the CI.
Then, re-embark the pull request into the merge queue by posting the comment
@mergifyio refresh on the pull request.

@michaelsproul michaelsproul added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Aug 2, 2024
@michaelsproul
Copy link
Member

@mergify refresh

Copy link

mergify bot commented Aug 2, 2024

refresh

✅ Pull request refreshed

@michaelsproul
Copy link
Member

@mergify queue

Copy link

mergify bot commented Aug 2, 2024

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at 0e96d4f

@mergify mergify bot merged commit 0e96d4f into sigp:unstable Aug 2, 2024
28 checks passed
chong-he pushed a commit to chong-he/lighthouse that referenced this pull request Nov 26, 2024
* Store changes to persist data columns.

Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com>

* Update to use `eip7594_fork_epoch` for data column slot in Store.

* Fix formatting.

* Merge branch 'unstable' into data-columns-store

# Conflicts:
#	beacon_node/store/src/lib.rs
#	consensus/types/src/chain_spec.rs

* Minor refactor.

* Merge branch 'unstable' into data-columns-store

# Conflicts:
#	beacon_node/store/src/metrics.rs

* Init data colum info at PeerDAS epoch instead of Deneb fork epoch. Address review comments.

* Remove Deneb-related comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
das Data Availability Sampling das-unstable-merge ready-for-merge This PR is ready to merge.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants