Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

switch to upstream gossipsub #7057

Merged
merged 5 commits into from
Mar 11, 2025
Merged

Conversation

jxs
Copy link
Member

@jxs jxs commented Feb 27, 2025

Issue Addressed

We forked gossipsub into the lighthouse repo sometime ago so that we could iterate quicker on implementing back pressure and IDONTWANT.
Meanwhile we have pushed all our changes upstream and we are now the main maintainers of rust-libp2p this allows us to use upstream gossipsub again.
Nonetheless we still use our forked repo to give us freedom to experiment with features before submitting them upstream

@jxs jxs requested review from ackintosh and AgeManning February 27, 2025 22:13
@jxs jxs force-pushed the sync-gossipsub-upstream branch 4 times, most recently from 7e2e2cd to 600cfc5 Compare February 28, 2025 10:57
@ackintosh
Copy link
Member

Thanks for this PR! I'm checking if there are any improvements from our fork that might have been missed during upstreaming.

Comment on lines -820 to -830
match self.gossipsub_mut().unsubscribe(&libp2p_topic) {
Err(_) => {
warn!(self.log, "Failed to unsubscribe from topic"; "topic" => %libp2p_topic);
false
}
Ok(v) => {
// Inform the network
debug!(self.log, "Unsubscribed to topic"; "topic" => %topic);
v
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did we remove these logs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cause we no longer err on unsubscribe() it always succeeds at the Behavior level

Comment on lines -915 to -921
if let Err(e) = self.gossipsub_mut().report_message_validation_result(
self.gossipsub_mut().report_message_validation_result(
&message_id,
propagation_source,
validation_result,
) {
warn!(self.log, "Failed to report message validation"; "message_id" => %message_id, "peer_id" => %propagation_source, "error" => ?e);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And these ones?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same reason, we no longer fail at this level, message validation report always succeeds

Comment on lines -1259 to -1265
if let Err(e) = self.gossipsub_mut().report_message_validation_result(
self.gossipsub_mut().report_message_validation_result(
&id,
&propagation_source,
MessageAcceptance::Reject,
) {
warn!(self.log, "Failed to report message validation"; "message_id" => %id, "peer_id" => %propagation_source, "error" => ?e);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here also

@jxs jxs force-pushed the sync-gossipsub-upstream branch from 600cfc5 to ce2b347 Compare March 3, 2025 20:18
Copy link
Member

@AgeManning AgeManning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, cool. LGTM

@ackintosh
Copy link
Member

I noticed that #6873 has not been upstreamed, but I'm not sure if we still need these metrics. cc: @pawanjay176

@pawanjay176
Copy link
Member

Thanks @ackintosh . I think one of the metrics in the PR is buggy, been meaning to fix it but haven't gotten around to it so its good that it doesn't makes it upstream.

@ackintosh
Copy link
Member

@jxs @pawanjay176
These metrics have also not been upstreamed. They looks useful to me, what do you think? I can submit a PR to upstream them if they're good.

/// The number of bytes we have received in every IDONTWANT control message.
idontwant_bytes: Counter,
/// Number of IDONTWANT messages sent per topic.
idontwant_messages_sent_per_topic: Family<TopicHash, Counter>,
/// Number of full messages we received that we previously sent a IDONTWANT for.
idontwant_messages_ignored_per_topic: Family<TopicHash, Counter>,

@jxs
Copy link
Member Author

jxs commented Mar 6, 2025

@pawanjay176 @ackintosh wdyt of adding them temporarily on sigp/rust-libp2p#570 until we confirm IDONTWANTs are saving us bandwidth?

@ackintosh
Copy link
Member

That makes sense to me.

Copy link

mergify bot commented Mar 6, 2025

This pull request has merge conflicts. Could you please resolve them @jxs? 🙏

@jimmygchen jimmygchen added the ready-for-merge This PR is ready to merge. label Mar 11, 2025
mergify bot added a commit that referenced this pull request Mar 11, 2025
@mergify mergify bot merged commit f23f984 into sigp:unstable Mar 11, 2025
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Networking ready-for-merge This PR is ready to merge.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants