Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lcli command for manual rescue sync #5458

Merged
merged 10 commits into from
Aug 12, 2024
Merged

Conversation

michaelsproul
Copy link
Member

Proposed Changes

Add an lcli rescue subcommand which can be used to manually sync blocks from one beacon node to another. I've been cheekily using to muck around on Goerli in the presence of non-finality, streaming blocks from the public Nimbus endpoint to ours (with seemingly no impact on the head).

@dapplion
Copy link
Collaborator

rescue is too generic of a name. Use http-sync or similar

@michaelsproul
Copy link
Member Author

michaelsproul commented Mar 25, 2024

good point

I might just close this PR for now, I'm not convinced it's worth merging. Was just chucking it here in case others wanted to use it

@dapplion
Copy link
Collaborator

The tool has value IMO, it's a plausible sync method if p2p breaks down I'm okay with merging it

@jimmygchen
Copy link
Member

Yep agree with Lion, I was desperately looking for this last night - looks like we might need it again 🙈

@jimmygchen jimmygchen reopened this Jul 8, 2024
@michaelsproul michaelsproul added work-in-progress PR is a work-in-progress and removed do-not-merge labels Jul 8, 2024
# Conflicts:
#	lcli/src/main.rs
@jimmygchen
Copy link
Member

Merged latest unstable into this branch

lcli/src/rescue.rs Outdated Show resolved Hide resolved
lcli/src/rescue.rs Outdated Show resolved Hide resolved
lcli/src/main.rs Outdated
@@ -552,6 +553,58 @@ fn main() {
.display_order(0)
)
)
.subcommand(
Command::new("rescue")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth adding a note that this tool only works if the distance is within the data availability windows (4096 epochs ~ 18 days) otherwise blobs will become unavailable.

for (slot, block) in blocks.iter().rev() {
println!("posting block at slot {slot}");
if let Err(e) = target.post_beacon_blocks(block).await {
println!("error posting {slot}: {e:?}");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is a non 200 status code, we should just break out of the loop.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it could be a 202 (or other error) for block-is-already-known

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but actually, that probably shouldn't happen, so yeah, I agree

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep It was a 202, I added a check for this

@jimmygchen jimmygchen added ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Jul 9, 2024
@jimmygchen
Copy link
Member

I've address the comments and added two optional flags --block-cache-dir and --known-common-ancestor.
PR is ready for a re-review of the new changes.

Copy link
Member

@jimmygchen jimmygchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaelsproul I've reviewed your earlier changes. Feel free to mark this as ready to merge once you've review my recent changes!

@jimmygchen jimmygchen added the lcli Relates to the lighthouse cli binary label Aug 9, 2024
Copy link
Collaborator

@dapplion dapplion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@jimmygchen jimmygchen added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Aug 12, 2024
@jimmygchen
Copy link
Member

@mergify queue

Copy link

mergify bot commented Aug 12, 2024

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at 781c5ec

mergify bot added a commit that referenced this pull request Aug 12, 2024
@mergify mergify bot merged commit 781c5ec into sigp:unstable Aug 12, 2024
28 checks passed
ThreeHrSleep pushed a commit to ThreeHrSleep/lighthouse that referenced this pull request Aug 12, 2024
* Rescue CLI

* Allow tweaking start block

* More caching

* Merge branch 'unstable' into rescue-cli

# Conflicts:
#	lcli/src/main.rs

* Add `--known–common-ancestor` flag to optimise for download speed.

* Rename rescue command to `http-sync`

* Add logging

* Add optional `--block-cache-dir` cli arg and create directory if it doesn't already exist.

* Lint fix.

* Merge branch 'unstable' into rescue-cli
AgeManning pushed a commit to AgeManning/lighthouse that referenced this pull request Sep 3, 2024
* Rescue CLI

* Allow tweaking start block

* More caching

* Merge branch 'unstable' into rescue-cli

# Conflicts:
#	lcli/src/main.rs

* Add `--known–common-ancestor` flag to optimise for download speed.

* Rename rescue command to `http-sync`

* Add logging

* Add optional `--block-cache-dir` cli arg and create directory if it doesn't already exist.

* Lint fix.

* Merge branch 'unstable' into rescue-cli
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lcli Relates to the lighthouse cli binary ready-for-merge This PR is ready to merge.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants