Ephemeral plugin traffic needs a way to reach a peer that does not share a backpressure domain with
durable replication. Most of the machinery now exists; what is missing is that it is closed to a
second consumer.
Why not multiplex onto replication
Two reasons, and the second is the real one.
Lifecycle. Connections are keyed per (peer, database), and a plugin channel is not a database.
The generic per-call path opens a fresh short-lived WebSocket per call, which is fine for an
occasional remote operation and unusable per message.
Backpressure. Putting ephemeral traffic on a replication socket puts a publish burst
head-of-line ahead of audit frames and pings on that connection. So a plugin workload could delay,
or past a ping timeout force a reconnect of, durable replication convergence. Ephemeral traffic must
have its own bounded queue whose drops cannot affect replication.
What already exists and should be reused
-
The RPC abstraction, but not the replication socket underneath it. Record locks reach peers
through registered operations that prefer this worker's live replication subscription session,
falling back to a per-call connection (replication/recordLockRpc.ts). The registered-operation
shape is reusable; its preference for the replication session is not.
Planning review caught this as a contradiction in an earlier draft of this issue, which demanded
an independent backpressure domain while proposing to generalize a path that rides the
replication leg. Those are incompatible, for exactly the reason given above: a publish burst on
that session delays audit frames and pings. Take the abstraction, give ephemeral traffic its own
connection and its own bounded queue.
-
Capability negotiation. Levels are advertised in the node-name capability bag in
replication/protocolCapabilities.ts. A plugin relay registers a level there. This retires
the original version-skew ask, and with it the concern that the inbound dispatch switch has no
default case.
-
The safety property. Control bytes above 127 stay off the transaction-replay path, which is
one check. A new non-durable message class provably cannot perturb replay.
What must be built
- Channel interest and drop accounting on a separate connection. This is the item that matches
the invariant, and it is the bulk of the work. Planning review pushed back on an earlier draft
that called opening the record-lock operation union "the cheap part": opening the union is a
prerequisite for a second consumer, but it is not a smaller version of this option, and
capability negotiation does not change backpressure. Do not let the cheap item stand in for the
real one.
- Open the closed union as well. The record-lock operation type is a closed union of four
operations and the transport interface core validates is a fixed, lock-specific method set, so a
second consumer cannot use either today.
- A channel or topic concept. Subscriptions are database and table filter sets throughout. A
free-form channel namespace is new, and so is interest propagation. Without interest propagation
every publish floods every node, and pattern subscriptions are harder still.
- A unified connected-peers accessor, reduced from the existing node map and connection maps.
- Bounded queues with drop accounting, so an endpoint can report drops rather than hide them.
Delivery guarantees to state up front
Redis pub/sub is at-most-once and drops for slow or disconnected subscribers, which is a lower
bar than anything else Harper does. Match it deliberately rather than accidentally building
something more durable and slower. Replication pauses under backpressure and replays after
reconnect; both are wrong for ephemeral messages, not merely slow.
Ephemeral plugin traffic needs a way to reach a peer that does not share a backpressure domain with
durable replication. Most of the machinery now exists; what is missing is that it is closed to a
second consumer.
Why not multiplex onto replication
Two reasons, and the second is the real one.
Lifecycle. Connections are keyed per (peer, database), and a plugin channel is not a database.
The generic per-call path opens a fresh short-lived WebSocket per call, which is fine for an
occasional remote operation and unusable per message.
Backpressure. Putting ephemeral traffic on a replication socket puts a publish burst
head-of-line ahead of audit frames and pings on that connection. So a plugin workload could delay,
or past a ping timeout force a reconnect of, durable replication convergence. Ephemeral traffic must
have its own bounded queue whose drops cannot affect replication.
What already exists and should be reused
The RPC abstraction, but not the replication socket underneath it. Record locks reach peers
through registered operations that prefer this worker's live replication subscription session,
falling back to a per-call connection (
replication/recordLockRpc.ts). The registered-operationshape is reusable; its preference for the replication session is not.
Planning review caught this as a contradiction in an earlier draft of this issue, which demanded
an independent backpressure domain while proposing to generalize a path that rides the
replication leg. Those are incompatible, for exactly the reason given above: a publish burst on
that session delays audit frames and pings. Take the abstraction, give ephemeral traffic its own
connection and its own bounded queue.
Capability negotiation. Levels are advertised in the node-name capability bag in
replication/protocolCapabilities.ts. A plugin relay registers a level there. This retiresthe original version-skew ask, and with it the concern that the inbound dispatch switch has no
default case.
The safety property. Control bytes above 127 stay off the transaction-replay path, which is
one check. A new non-durable message class provably cannot perturb replay.
What must be built
the invariant, and it is the bulk of the work. Planning review pushed back on an earlier draft
that called opening the record-lock operation union "the cheap part": opening the union is a
prerequisite for a second consumer, but it is not a smaller version of this option, and
capability negotiation does not change backpressure. Do not let the cheap item stand in for the
real one.
operations and the transport interface core validates is a fixed, lock-specific method set, so a
second consumer cannot use either today.
free-form channel namespace is new, and so is interest propagation. Without interest propagation
every publish floods every node, and pattern subscriptions are harder still.
Delivery guarantees to state up front
Redis pub/sub is at-most-once and drops for slow or disconnected subscribers, which is a lower
bar than anything else Harper does. Match it deliberately rather than accidentally building
something more durable and slower. Replication pauses under backpressure and replays after
reconnect; both are wrong for ephemeral messages, not merely slow.