Execution meta — see plan: #5296
Problem
EventMeshApplication.enableCluster() configures the sticky membership model and creates ClusterMembership, but does not visibly start a periodic membership heartbeat. The periodic call to ClusterMembership.heartbeat() is currently driven by PartitionOwnership.refresh(), while the sticky path intentionally does not wire PartitionOwnership.
As a result, the default sticky-cluster path may not publish/refresh /em/instances/<id> membership records. This affects /session/recommend, load-based instance selection, stale-member expiry, and graceful leave semantics.
Relevant code:
eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshApplication.java: enableCluster()
eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/cluster/ClusterMembership.java: heartbeat()
eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/cluster/PartitionOwnership.java: heartbeat scheduling in refresh()
Reproduction
- Start
EventMeshApplication with clustered Meta configuration.
- Call
enableCluster() but do not configure PartitionOwnership.
- Inspect MetaStore for
/em/instances/<instanceId> over multiple TTL intervals.
- Query
/session/recommend from another instance.
- Observe whether membership and load information is published and refreshed.
Expected behavior
Every cluster topology that depends on membership must own an explicit membership lifecycle: immediate startup heartbeat, periodic refresh before TTL expiry, health/degraded behavior when Meta is unavailable, and deletion on graceful shutdown.
Proposed direction
- Introduce a dedicated membership lifecycle service/scheduler independent of partition ownership.
- Start it from
enableCluster() for sticky mode.
- Stop it before
leave() during shutdown.
- Define Meta outage behavior for readiness and
/session/recommend.
Acceptance criteria
Suggested labels
bug, reliability, cluster, priority:p0
Problem
EventMeshApplication.enableCluster()configures the sticky membership model and createsClusterMembership, but does not visibly start a periodic membership heartbeat. The periodic call toClusterMembership.heartbeat()is currently driven byPartitionOwnership.refresh(), while the sticky path intentionally does not wirePartitionOwnership.As a result, the default sticky-cluster path may not publish/refresh
/em/instances/<id>membership records. This affects/session/recommend, load-based instance selection, stale-member expiry, and graceful leave semantics.Relevant code:
eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/boot/EventMeshApplication.java:enableCluster()eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/cluster/ClusterMembership.java:heartbeat()eventmesh-runtime/src/main/java/org/apache/eventmesh/runtime/cluster/PartitionOwnership.java: heartbeat scheduling inrefresh()Reproduction
EventMeshApplicationwith clustered Meta configuration.enableCluster()but do not configurePartitionOwnership./em/instances/<instanceId>over multiple TTL intervals./session/recommendfrom another instance.Expected behavior
Every cluster topology that depends on membership must own an explicit membership lifecycle: immediate startup heartbeat, periodic refresh before TTL expiry, health/degraded behavior when Meta is unavailable, and deletion on graceful shutdown.
Proposed direction
enableCluster()for sticky mode.leave()during shutdown./session/recommend.Acceptance criteria
/session/recommendonly uses live membership records.PartitionOwnership, TTL expiry, Meta outage, and shutdown.Suggested labels
bug,reliability,cluster,priority:p0