Cluster Subscriptions with Topology Changes

[jaredpetersen]

I'd like to subscribe to key-space notifications using Redis Pub/Sub on a cluster.

What do I need to do to ensure that new nodes being added to the Redis cluster are also subscribed to?

According to the documentation, this can be a bit of a gotcha because:

Subscription registrations are not propagated to new nodes that are added on a topology change

Since key-space notifications are node-local, I have this sort of setup:

RedisClusterClient clusterClient = RedisClusterClient.create("someuri");
clusterClient.setOptions(ClusterClientOptions.builder()
    .topologyRefreshOptions(ClusterTopologyRefreshOptions.builder()
      .enableAllAdaptiveRefreshTriggers()
      .enablePeriodicRefresh()
      .build())
    .build());

StatefulRedisClusterPubSubConnection<String, String> pubSubConnection = clusterClient.connectPubSub();
pubSubConnection.setNodeMessagePropagation(true);

Is there some kind of Topology hook that I can tap into so that the new nodes get the subscription (and Lettuce listens to them)? Is there something I need to do with the NodeSelection API?

And secondarily, what's a good way to ensure that you only subscribe to the master cluster nodes (while again keeping up with topology changes) to avoid duplication issues?

Subscribing to non-master nodes, therefore, will let your application see multiple events of the same type for the same key because of Redis distributed nature

If it was just StatefulRedisClusterConnection you would just just call setReadFrom(ReadFrom.UPSTREAM) but there's nothing like that for the pub/sub connection.

[mp911de]

Cluster Keyspace Notifications are hard to do right because a cluster changes its topology without a reliable source for change notifications. The only mechanism is polling for changes.

Is there some kind of Topology hook

Lettuce emits ClusterTopologyChangedEvent through its event bus. You can subscribe to ClientResources.eventBus().get() to listen for these events.

And secondarily, what's a good way to ensure that you only subscribe to the master cluster nodes

You can either subscribe through the NodeSelection API or track nodes yourself by inspecting Partitions.

Note that ReadFrom helps to route commands to replicas/master nodes, however keyspace notifications are active push messages from the Redis server and ReadFrom has no effect there.