feat: rendezvous

[salmad3]

Context

• Adds content for libp2p rendezvous
• Part of Create and populate discovery & routing section #250

Notes

Section this document lives under will become populated with corresponding PRs.

Latest preview

Please view the latest Fleek preview here.

[mxinden]

Thanks for the write up @DannyS03. A couple of comments.

Maybe @thomaseizinger, as the author of the rust-libp2p implementation and specification, can you take a look?

[mxinden]

Is that implemented anywhere?

[salmad3]

Not aware of an implementation myself, but it is possible (as per the spec).

[thomaseizinger]

Thanks for the ping @mxinden.

Before I comment on any details of the text, I'd like to point out that to my knowledge and compared to other discovery protocols like the DHT, rendezvous is not actually used by IPFS or other large deployment of libp2p.

For example, the implementation of the protocol in Go lives outside the main repository: https://github.com/berty/go-libp2p-rendezvous.
A search for "rendezvous" in the go-libp2p repository only yields (slightly misleading) examples: https://github.com/search?q=repo%3Alibp2p%2Fgo-libp2p+rendezvous+language%3AGo&type=code&l=Go

I am not sure what the scope of this recent docs update is but in my opinion, it would be better to:

• Update the rendezvous spec to remove the sections about "replacing ws-star-rendezvous" and other things that have never been implemented.
• Overall revise the spec to more clearly state what it is without any of the "we may do X, Y or Z" with this.
• Trim this document down to a high-level introduction
• Link to the spec for any details

Compared to the similar discussion we had in the TLS PR, we actually did invent this. Consequently, there isn't much we can say here in regards to "This is the general rendezvous protocol and libp2p uses only this bit". Instead, I think we should just give a brief introduction and then link to the spec for any details. If we give the spec a slight polish, it should be easier to approach which would be a net-win because it also benefits people that discover the spec from somewhere else other than this docs site :)

My 2c. Let me know what you guys think!

[marten-seemann]

I agree with the points made about the use of "real-time".

I also think we need to point out that rendezvouz is not decentralized. It's federated at best. That's not necessarily a bad thing, there are valid use cases for that, but it introduces one central point of failure into the network. This description can then be used to contrast rendezvouz with fully decentralized solutions like the DHT and Gossipsub.

It would also be valuable to point out that rendezvouz is not used by IPFS and Filecoin (where is it used at all?).

[thomaseizinger]

It would also be valuable to point out that rendezvouz is not used by IPFS and Filecoin (where is it used at all?).

We used it at my old job for a project which I think is still live (that is why I initially wrote the Rust implementation).

nim-libp2p has an implementation but I don't know about any deployments. (cc @Menduist)

The go implementation I linked above is from berty but I heard that is a forked version and thus may not be compatible? (cc @gfanton)

[Menduist]

We currently don't use it, but probably will in the next few months.

I think RendezVous is one of the best "service discovery" method we can have
It's also more suited to mobiles & co

I also think it does not has to be centralized: in our implementation, everyone is storing records, and when pulling records from a peer, you will also store them. So sure, you need "centralized bootnodes", exactly like the DHT
But once you discovered other peers, you can just exchange SPRs in a decentralized manner.

IMO it's just a different use-case from the DHT. A DHT is very powerful for:

• Finding a specific value in the network
• Random walks

But when you want to find a "group of peers" (for instance for service discovery), you either need to random-walk your way to them, or store every peers of that group under a single ID (which puts them in a very limited amount of nodes). None of which is ideal

Happy to be corrected here :) We'll try to do some simulations in the upcoming months to see if we can get to a fully decentralized rendez-vous giving better perfs than DHT for service discovery.

[gfanton]

We use a fork of an implementation made by https://github.com/vyzo (https://github.com/libp2p/go-libp2p-rendezvous/tree/implement-spec) at Berty, and it has been working well for us. I'm not sure how compatible it is with the nim-libp2p implementation though. The main change we made was to improve the Subscribe method, which allowed us to add MQTT servers for real-time discovery notification.

As @Menduist mentioned, the Rendezvous service is especially useful on mobile devices where applications have short-lived sessions and cannot use the DHT efficiently due to factors such as bootstrap time and discovery latency. We created a multi-discovery system that advertises on both the DHT and the rendezvous to provide a fallback option, but I'm not completely satisfied with it because it relies too much on the rendezvous and doesn't act as a truly decentralized service. Additionally, the DHT is essentially meaningless in this setup because it is hidden by the rendezvous service and uses too much energy on mobile devices.

Happy to be corrected here :) We'll try to do some simulations in the upcoming months to see if we can get to a fully decentralized rendez-vous giving better perfs than DHT for service discovery.

Please keep us in touch, I'm really looking forward to this 😄

[mxinden]

This is good to go from my end @p-shahi. What do you suggest as next steps? I don't understand the branching strategy here.