Skip to content

Add support for sharded PubSub #2373

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 57 commits into from
Jan 25, 2023
Merged

Add support for sharded PubSub #2373

merged 57 commits into from
Jan 25, 2023

Conversation

leibale
Copy link
Contributor

@leibale leibale commented Jan 10, 2023

Refactor PubSub
fix #2354 - Add support for sharded PubSub
fix #1968 - PUBSUB SHARDCHANNELS command
fix #2345 ref #2321 - PING in PubSub mode
fix #2302 - error listener note in the main README
fix #2065 - Cluster + PubSub
fix #1973 - SSUBSCRIBE
fix #1975 - SUNSUBSCRIBE
Add minimizeConnections configuration for clusters

renamed branch from #2344

@leibale leibale marked this pull request as draft January 10, 2023 01:07
@codecov-commenter
Copy link

codecov-commenter commented Jan 10, 2023

Codecov Report

Base: 95.85% // Head: 95.83% // Decreases project coverage by -0.02% ⚠️

Coverage data is based on head (a94d316) compared to base (6642278).
Patch coverage: 83.10% of modified lines in pull request are covered.

❗ Current head a94d316 differs from pull request most recent head 80eddda. Consider uploading reports for the commit 80eddda to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2373      +/-   ##
==========================================
- Coverage   95.85%   95.83%   -0.02%     
==========================================
  Files         451      454       +3     
  Lines        4247     4515     +268     
  Branches      475      511      +36     
==========================================
+ Hits         4071     4327     +256     
- Misses        106      115       +9     
- Partials       70       73       +3     
Impacted Files Coverage Δ
packages/client/lib/commands/CLUSTER_SLOTS.ts 100.00% <ø> (ø)
...ckages/client/lib/commands/generic-transformers.ts 100.00% <ø> (ø)
packages/client/lib/cluster/index.ts 80.00% <76.92%> (+2.38%) ⬆️
packages/client/lib/cluster/cluster-slots.ts 80.00% <79.80%> (+20.54%) ⬆️
packages/client/lib/client/pub-sub.ts 82.73% <82.73%> (ø)
packages/client/lib/client/commands-queue.ts 88.42% <90.24%> (+5.52%) ⬆️
packages/client/lib/client/socket.ts 82.30% <92.00%> (+4.34%) ⬆️
packages/client/lib/client/index.ts 92.60% <95.23%> (+0.60%) ⬆️
packages/client/lib/client/commands.ts 100.00% <100.00%> (ø)
packages/client/lib/cluster/commands.ts 100.00% <100.00%> (ø)
... and 9 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@leibale leibale marked this pull request as ready for review January 23, 2023 22:06
@leibale leibale merged commit 3b1bad2 into redis:master Jan 25, 2023
@leibale leibale deleted the sharded-pubsub branch January 25, 2023 18:36
florian-schunk pushed a commit to florian-schunk/node-redis that referenced this pull request Jun 18, 2025
* refactor pubsub, add support for sharded pub sub

* run tests in redis 7 only, fix PUBSUB SHARDCHANNELS test

* add some comments and fix some bugs

* PubSubType, not PubSubTypes 🤦‍♂️

* remove test.txt

* fix some bugs, add tests

* add some tests

* fix redis#2345 - allow PING in PubSub mode (remove client side validation)

* remove .only

* revert changes in cluster/index.ts

* fix tests minimum version

* handle server sunsubscribe

* add 'sharded-channel-moved' event to docs, improve the events section in the main README (fix redis#2302)

* exit "resubscribe" if pubsub not active

* Update commands-queue.ts

* Release client@1.5.0-rc.0

* WIP

* use `node:util` instead of `node:util/types` (to support node 14)

* run PubSub resharding test with Redis 7+

* fix inconsistency in live resharding test

* add some tests

* fix iterateAllNodes when starting from a replica

* fix iterateAllNodes random

* fix slotNodesIterator

* fix slotNodesIterator

* clear pubSubNode when node in use

* wait for all nodes cluster state to be ok before testing

* `cluster.minimizeConections` tests

* `client.reconnectStrategry = false | 0` tests

* sharded pubsub + cluster 🎉

* add minimum version to sharded pubsub tests

* add cluster sharded pubsub live reshard test, use stable dockers for tests, make sure to close pubsub clients when a node disconnects from the cluster

* fix "ssubscribe & sunsubscribe" test

* lock search docker to 2.4.9

* change numberOfMasters default to 2

* use edge for bloom

* add tests

* add back getMasters and getSlotMaster as deprecated functions

* add some tests

* fix reconnect strategy + docs

* sharded pubsub docs

* Update pub-sub.md

* some jsdoc, docs, cluster topology test

* clean pub-sub docs

Co-authored-by: Simon Prickett <simon@redislabs.com>

* reconnect startegy docs and bug fix

Co-authored-by: Simon Prickett <simon@redislabs.com>

* refine jsdoc and some docs

Co-authored-by: Simon Prickett <simon@redislabs.com>

* I'm stupid

* fix cluster topology test

* fix cluster topology test

* Update README.md

* Update clustering.md

* Update pub-sub.md

Co-authored-by: Simon Prickett <simon@redislabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment