Rework PubSub and fix race conditions (#27)
- add
aioredis_cluster.aioredis.stream
module - rework PubSub command execution flow for prevent race conditions on spontaneously server channels unsubscribe push
- make fully dedicated
RedisConnection
implementation for cluster RedisConnection
once entered in PubSub mode was never exit in them, because is too hard handle spontaneously unsubscribe events from Redis with simultaneously(P|S)UNSUBSCRIBE
manually calls- fully rewrite handling PUB/SUB replies/events
- for
Cluster
,RedisConnection
andConnectionsPool
in_pubsub
indicates flag when connector have in pubsub mode connections instead number of PUB/SUB channels - add key slot handling for sharded PubSub channels in non-cluster dedicate
RedisConnection
- fix and improve legacy
aioredis
tests - improve support for py3.12
- improve support for Redis 7.2
- fix stuck
aioredis.Connection
socket reader routine for sharded PUB/SUB when cluster reshard and Redis starts respondMOVED
error onSSUBSCRIBE
commands #24
- improve connection creation timeout
- do not lose connection in Pool while execute PING probe
- respect Pool.minsize in idle connections detector
- shuffle startup nodes for obtain cluster state
- add support Sharded PUB/SUB
- new methods and properties
spublish
,ssubscribe
,sunsubscribe
,pubsub_shardchannels
,pubsub_shardnumsub
,sharded_pubsub_channels
- drop support Python 3.6, 3.7
- add support Python 3.11
- idle connections detection in connections pool
- change acquire connection behaviour from connection pool. Now connection acquire and release to pool by LIFO way for better idle connection detection
- deprecated
state_reload_frequency
option fromcreate_cluster
factory was removed
- fix bypass
username
argument for pool creation
- add support Redis 6
AUTH
command with username - factories
create_cluster
,create_redis_cluster
,aioredis_cluster.aioredis.create_connection
now supportusername
argument - add
auth_with_username
method forAbcConnection
,AbcPool
and impementations
- fix problem when RedisConnection was GC collected after unhandled
asyncio.CancelledError
- fix default
db
argument for pool/connection in cluster mode
- (revert) apply cluster state only if cluster metadata is changed
- fetch several cluster state candidates from cluster for choose best metadata for final local state
- apply cluster state only if cluster metadata is changed
- FIX: handle closed pubsub connection before gc its collected that trigger
Task was destroyed but it is pending!
message in log - improve logging in state loader
- fix bug when
ConnectionsPool.acquire()
is stuck because closed PUB/SUB connection is not cleanup fromused
set - fix
ConnectionsPool.acquire()
incorrect wakeup order for connection waiters when connection is released ConnectionsPool.execute()
now acquire dedicate connection for execution if command is blocking, ex.BLPOP
ConnectionsPool.execute()
now raisesValueError
exception for PUB/SUB family command- In
ConnectionsPool
PUB/SUB dedicated connections now is closing onclose()
call - add
aioredis_cluster.abc.AbcConnection
abstract class - add property
readonly
and methodset_readonly()
foraioredis_cluster.abc.AbcConnection
andaioredis_cluster.abc.AbcPool
aioredis_cluster.Cluster
now requirepool_cls
implementation fromaioredis_cluster.abc.AbcPool
- add
ssl
argument for factoriescreate_cluster
,create_redis_cluster
andCluster
constructor - add 10% jitter for cluster state auto reload interval
- fix incorrect iterate free connections in
select()
,auth()
methods forConnectionsPool
- include
aioredis==1.3.1
source code intoaioredis_cluster._aioredis
and introduceaioredis_cluster.aioredis
but for compatible and migration period - this release have not backward incompatible changes
- DEPRECATION WARNING: you must migrate from
import aioredis
toimport aioredis_cluster.aioredis
becauseaioredis_cluster
starts vendorizeaioredis
package and maintain it separately. Usingaioredis
package will be removed in v3 - fix reacquire connection in
aioredic.ConnectionsPool
after Redis node failure
- Add
xadd_620
commands method for supportXADD
options for Redis 6.2+
- add
ClusterState.slots_assigned
- require reload cluster state for some cases with
UncoveredSlotError
- add
execute_timeout
forManager
- improve cluster state reload logging
- reduce number of addresses to fetch cluster state
- acquire dedicate connection from pool to fetch cluster state
- extend
ClusterState
by new attributes:state
,state_from
,current_epoch
- fix keys extraction for
XREAD
andXREADGROUP
commands
- make public
Address
,ClusterNode
andClusterState
structs. Available by importfrom aioredis_cluster import
Cluster
provides some new helpful methods:get_master_node_by_keys(*keys)
- return masterClusterNode
which contains keyskeys
create_pool_by_addr(addr, **kwargs)
- create connection pool byaddr
and return pool wrapped bycommands_factory
fromCluster
constructor. By default isaioredis_cluster.RedisCluster
instance.get_cluster_state()
- returnClusterState
instance with recent known cluster state received from Redis clusterextract_keys(command_sequence)
- returns keys of command sequence
- drop
pytest-aiohttp
plugin for tests - add
pytest-asyncio
dependency for tests - switch
asynctest
->mock
library for aio tests - drop
attrs
dependency. For Python 3.6 you need installdataclasses
- fix extract keys for
BLPOP
/BRPOP
commands - add support keys extraction for
ZUNION
,ZINTER
,ZDIFF
,ZUNIONSTORE
,ZINTERSTORE
,ZDIFFSTORE
commands - acquire dedicate connection from pool for potential blocking commands like
BLPOP
,BRPOP
,BRPOPLPUSH
,BLMOVE
,BLMPOP
,BZPOPMIN
,BZPOPMAX
,XREAD
,XREADGROUP
- README update
- speedup crc16. Use implementation from python stdlib
- remove
state_reload_frequency
fromClusterManager
.state_reload_interval
now is one relevant option for state auto reload - default
state_reload_interval
increased and now is 300 seconds (5 minutes) - commands registry loads only once, on cluster state initialize
- improve failover. First connection problem cause retry to random slot replica
- improve python3.9 support
- default
idle_connection_timeout
now is 10 minutes
- fix
aioredis.locks.Lock
issue (aio-libs-abandoned/aioredis-py#802, bpo32734) - now
aioredis_cluster.Cluster
do not acquire dedicate connection for every execute aioredis_cluster
now requirespython>=3.6.5
- improve compatible with Python 3.8
- improve failover logic while command timed out
- read-only commands now retries if attempt_timeout is reached
- add required dependeny
async_timeout
aioredis
dependency bound now isaioredis >=1.1.0, <2.0.0
- add timeout for command execution (per execution try)
- add Cluster option
attempt_timeout
for configure command execution timeout, default timeout is 5 seconds - Cluster.execute_pubsub() fixes
- CHANGES fix
- Cluster state auto reload
- new
state_reload_frequency
option to configure state reload frequency - new
state_reload_interval
option to configure state auto reload interval follow_cluster
option enable load cluster state from previous cluster state nodes- establish connection only for master nodes after cluster state load
- change default commands_factory to aioredis_cluster.RedisCluster instead aioredis.Redis
- all cluster info commands always returns structs with str, not bytes
keys_master
andall_masters
methods now try to ensure cluster state instead simply raise exception if connection lost to cluster node, for examplemax_attempts
always defaults fix
- Library full rewrite
- Cluster state auto reload
- Command failover if cluster node is down or key slot resharded
- Pipeline and MULTI/EXEC cluster implementation with keys distribution limitation (because cluster)
- Python 3.6+ only
- Initial release based on aioredis PR (aio-libs-abandoned/aioredis-py#119)