Skip to content

[Architecture Review][P0] Configurable DeliveryTopology for cluster delivery #5300

Description

@qqeasonchen

Problem

Two cluster delivery models coexist. EventMeshApplication.enableCluster() uses the sticky local-delivery model and intentionally does not enable partition ownership or cross-instance forwarding, yet the repository still retains PartitionOwnership, ClusterCoordinator, and HttpForwarder — a second partition-ownership / forwarding model. The partition-ownership code itself acknowledges missing generation fencing and incomplete remote offsets.

Risks

  • Documentation, code, and actual deployment behavior can diverge.
  • The sticky model can cause duplicate reads and amplification at the underlying MQ across instances.
  • The partition-ownership model lacks the fencing needed for safe handoff.

Proposed direction

Abstract the cluster mode into a configurable DeliveryTopology:

LOCAL_STICKY_PULL     current default: session pinned to an instance, local pull + local delivery
PARTITION_OWNED_PULL  high-throughput: single partition owner, requires fencing and handoff
FORWARDED_DISPATCH    compatibility: centralized pull + cross-instance forwarding

Each mode must document:

  • duplicate-delivery semantics;
  • cost / throughput characteristics;
  • failure-recovery strategy;
  • a corresponding E2E test.

Acceptance criteria

  • One explicit DeliveryTopology config selects the mode; inactive topology components are not half-wired.
  • Sticky and partition-ownership modes are documented with duplicate/cost/recovery semantics.
  • Partition handoff uses fencing (epoch/lease); a stale owner cannot keep delivering.
  • E2E tests exist per topology (Meta outage, network split, owner crash).

Related: #5293 (unify cluster delivery topology and fence stale partition owners).


Part of the Architecture Review.

Metadata

Metadata

Assignees

No one assigned

    Labels

    discussionThis issue requires further discussionimprovementImprove the mechanism or performance

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions