Repository navigation
feat(redis)!: native async cluster Pub/Sub on redis-py 8 - #3131
Merged
Merged
Conversation
Lancetnik
requested review from
IvanKirpichnikov and
powersemmi
as code owners
September 14, 2026 05:19
IvanKirpichnikov
requested changes
Sep 14, 2026
powersemmi
approved these changes
Sep 19, 2026
Lancetnik
force-pushed
the
feat/redis-cluster-native-async-pubsub
branch
from
September 23, 2026 05:18
ad65ad2 to
2275ac7
Compare
Lancetnik
added a commit
that referenced
this pull request
Sep 23, 2026
The five how-to pages send contributors to the navigation file by its GitHub URL on `main`. On 1.0.0 that file moved to `docs/docs/en/SUMMARY.md`, a path `main` gains only when this release ships, so lychee answers 404 and takes the `docs` gate with it. It went unnoticed because PR #3125 is a draft and every test job skips drafts; #3131 is the first non-draft PR on this base to run the check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
redis-py 8.0.0 gave the async `RedisCluster` `publish` and `pubsub`, so the synchronous workaround from #2854 can go: no second Redis client, no `ThreadPoolExecutor` in an otherwise async path, and no cluster-only code branch wherever Pub/Sub is involved. `RedisClusterFastProducer`, `_SyncPubSubProxy`, `sync_publish()`, `_get_sync_cluster()` and the `_sync_cluster` / `_thread_pool` fields on the base `ConnectionState` are gone. Cluster now uses the same `RedisFastProducer` and the stock `ChannelSubscriber.start()` as standalone, which also means every connection option — `credential_provider` for IAM / token auth included — reaches Channels, Lists and Streams through one client. BREAKING CHANGE: `faststream[redis]` now requires `redis>=8.0.0`. Closes #3009 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Lancetnik
force-pushed
the
feat/redis-cluster-native-async-pubsub
branch
from
September 23, 2026 05:25
2275ac7 to
16c4143
Compare
…tch auto-connect `_get_driver_info()` only existed to branch between `driver_info` and `lib_name`/`lib_version`; with the `redis>=8` floor it is a constant. `RedisClusterBroker.publish_batch` was the last publish path that connected implicitly — cluster `publish` and standalone `publish_batch` both require a started broker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IvanKirpichnikov
approved these changes
Sep 23, 2026
Lancetnik
added a commit
that referenced
this pull request
Sep 23, 2026
The instance-level `start` replacement goes away with the native async cluster Pub/Sub (#3131, on the 1.0.0 branch); the class and its `exclude-classes` entry say so. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Lancetnik
added a commit
that referenced
this pull request
Sep 23, 2026
Conflicts, resolved to 1.0.0's version: - `faststream/_internal/fastapi/*`: the FastAPI plugin is gone on 1.0.0; main only added `__slots__` there. - `faststream/redis/configs/state.py`, `faststream/redis/publisher/producer.py`: 1.0.0 dropped the sync cluster proxy and the separate cluster producer (#3131); main's `__slots__` (#3220) are kept on the classes that remain. - `pyproject.toml`: 1.0.0's `requires-python` and version.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3009. Targets
1.0.0rather thanmain, because it raises theredisfloor.What this removes
Redis Cluster Pub/Sub has been running through a synchronous client since #2854, because
redis.asyncio.clusterhad nopublish/pubsubat the time. redis-py 8.0.0 added them, so the workaround goes:_SyncPubSubProxy,sync_publish(),_get_sync_cluster()and theThreadPoolExecutorinconfigs/state.py— along with the_sync_cluster/_thread_poolfields that the baseConnectionStatecarried for the cluster's sakeRedisClusterFastProducerinpublisher/producer.py, whose channel publish hopped to a thread on every message_make_channel_subscriber()and thesubscriber()override inbroker/cluster_broker.py, which existed only to monkeypatchstart()on channel subscribersCluster now uses the same
RedisFastProducerand the stockChannelSubscriber.start()as standalone.BaseRedisFastProducercollapsed intoRedisFastProducersince there is no longer a second producer to share with.The
credential_providergap @fbartonoctopus reported on #3009 closes as a side effect: the hardcoded option whitelist in_get_sync_cluster()is gone, so every connection option reaches Channels, Lists and Streams through the one async client.The dependency decision
The issue left this open. This PR takes option 2 — raise the floor to
redis>=8.0.0— rather than the runtime guard #2893 used. One supported path, and it letsfaststream/redis/_compat.pygo entirely:REDIS_V710andREDIS_V720are now always true, so theclaim_min_idle_timeversion guard, thelib_name/lib_versionfallback in_get_driver_info(), and therequire_redis_v710test marker all disappear with it.This is breaking for every Redis user, not just cluster users — hence
1.0.0.One real bug the live cluster surfaced
ClusterPubSub.execute_commandreadsnodes_manager.slots_cachedirectly instead of going throughRedisCluster.execute_command, so it never triggers the client's lazy topology discovery. A channel subscriber is usually the first cluster operation an app performs, and it died onKeyError: <slot>.RedisClusterConnectionState.connect()now awaitsclient.initialize().That makes
connect()reach the network, which is what "connect" should mean — and it caught four tests that were only passing because the old connect was a no-op:test_start_stop_lifecycleranRedisClusterBroker()against the defaultlocalhost:6379with no cluster there; it is nowconnected-marked and points at the real clustertest_pipeline_warnspublished without starting the broker, which worked only because the sync client built itself lazily from_options; it now starts the broker, matching what standaloneRedisBrokeralready requiredtest_broker_gets_patched_attrs_within_cmandtest_broker_with_real_doesnt_get_patchedcome from the sharedBrokerTestclientTestcaseand start the broker for real. The cluster overrides carryconnectednow, which moves them fromtest-redis-cluster-smoke, where no cluster runs, totest-redis-cluster-realChecked during implementation
The issue flagged three things worth confirming, since #2893 predates current
main:legacy_responses=True— not needed. redis-py 8 already defaultsRedisCluster(legacy_responses=True), so RESP3 still yields RESP2-shaped responses and stream/list parsing is untouched. Passing it explicitly would be noise.driver_info) — already landed;_get_driver_info()just loses its pre-7.2 branch.Validation
ruff format/ruff check/codespell/mypy(1746 files) /pyrightcleantests/brokers/redis+tests/docs/redis, in-memory: passtests/brokers/redis -m redisagainst a live standalone Redis: passtests/brokers/redis/cluster -m redis_clusteragainst a live 3-node cluster (redis 8, ports 7001-7003): pass