Skip to content

add: mux overview #270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Feb 7, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 46 additions & 18 deletions content/concepts/multiplex/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,24 +9,52 @@ aliases:

## Overview

Stream Multiplexing (_stream muxing_) is a way of sending multiple streams of data over one
communication link. It combines multiple signals into one unified signal so it can be transported
'over the wires', then it is demulitiplexed (_demuxed_) so it can be output and used by separate
applications. This is done to share a single TCP connection using unique port numbers to distinguish
streams, between the multiple proceeses (such as kademlia and gossipsub) used by applications (such as IPFS)
to make connection and transmission more efficient.

With muxing, libp2p applications may have many separate streams of communication between peers, as well as
have multiple concurrent streams open at the same time with a peer. Stream multiplexing allows us to initialize
and use the same [transport](/concepts/transports/overview) connection across the lifetime of our interaction with a
peer. With muxing, we also only need to deal with [NAT traversal](/concepts/nat/overview) once to be able to open as
many streams as we need, since they will all share the same underlying transport connection. Applications can enable
support for multiple multiplexers, which will allow you to fall back to a widely-supported multiplexer if a preferred
choice is not supported by a remote peer.
libp2p is built on top of a stream abstraction and uses a bi-directional message stream to send data between peers.
However, relying on a single message
stream over a connection between two peers can result in scalability issues and bottlenecks.
Each peer on either side of the connection may run multiple applications sending and waiting for data over the stream.
A single stream would block applications on one another, as one application would
need to wait for another to finish utilizing the stream before being able to send
and receive its own messages.

To overcome this issue, libp2p enables applications to employ stream multiplexing.
Multiplexing allows
for the creation of multiple "virtual" connections within a single connection. This
enables nodes to send multiple streams of messages over separate virtual connections,
providing a scalable solution that eliminates the bottleneck created by a
single stream.
Two libp2p peers may have a single TCP connection and use different port numbers to distinguish streams.
Then different applications/processes like Kademlia or GossipSub used by an application like IPFS would get their own stream of data and make transmission more efficient.
Stream multiplexing makes it so that applications or protocols running on top of libp2p think that they’re the only ones running on that connection.
Another example is when HTTP/2 introduced streams into HTTP,
allowing for many HTTP requests in parallel on the same connection.

In summary, stream muxing can be used by applications on top of libp2p to share a single connection between various protocols,
providing a more efficient solution, particularly when establishing the connection is
resource-intensive, such as when NAT hole punching is necessary. By establishing a
connection once and running multiple streams over the same connection, libp2p can reduce
the resource overhead and latency penalty associated with frequent connection establishment.

## Stream Multiplexers in libp2p

Stream muxers are a key component of the libp2p stack, providing pluggable multiplexing
capabilities for peers. libp2p hosts can support multiple muxers simultaneously, and the
choice of muxer is negotiated between the nodes during the initial connection handshake.
This negotiation protocol allows libp2p to adopt new muxers in the future while
maintaining backward compatibility with existing muxers.

{{< alert icon="💡" context="info">}}
libp2p's multiplexing happens at the application layer, meaning it's not provided by the
operating system's network stack. However, developers writing libp2p applications rarely need to
interact with stream multiplexers directly, except during initial configuration to control which
modules are enabled.
For developers building libp2p applications, interaction with stream muxers is typically
limited to the initial configuration phase. The libp2p stack automatically handles the
negotiation and setup of the muxer, ensuring that all connections are stream-multiplexed
and allowing for the seamless transmission of multiple streams of data over a single
connection.
{{< /alert >}}

Currently, libp2p supports two stream muxers, [mplex](/concepts/multiplex/mplex)
and [yamux](/concepts/multiplex/yamux). However, many of the
[transport protocols](/concepts/transports/overview) available in the libp2p stack
come with native streams, such as [QUIC](/concepts/transports/quic),
[WebTransport](/concepts/transports/webtransport), and
[WebRTC](/concepts/transports/webrtc), and in these cases, libp2p
**does not need to perform stream multiplexing** as the protocol already provides it.