Skip to content

Conversation

vladvildanov
Copy link
Collaborator

Pull Request check-list

Please make sure to review and check all of these items:

  • Do tests and lints pass with this change?
  • Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Is there an example added to the examples folder (if applicable)?

NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.

Description of change

Please provide a description of the change here.

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds async implementation of MultiDBClient to support asynchronous operations with multiple Redis databases. The implementation provides circuit breaker patterns, health checks, and failure detection in an async context.

  • Replaces SyncCircuitBreaker with unified CircuitBreaker interface across all components
  • Adds complete async multidb module with client, configuration, command execution, and health checks
  • Extends background scheduler with async recurring task support

Reviewed Changes

Copilot reviewed 29 out of 32 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/test_multidb/test_config.py Updates circuit breaker imports and removes empty lines
tests/test_multidb/test_client.py Removes unnecessary blank lines in test assertions
tests/test_multidb/test_circuit.py Updates circuit breaker type references in imports and callbacks
tests/test_multidb/conftest.py Updates all mock circuit breaker references to unified type
tests/test_background.py Adds async test coverage for recurring background tasks
tests/test_asyncio/ Comprehensive test suite for async multidb components
redis/utils.py Adds async dummy failure function for retry operations
redis/multidb/ Updates circuit breaker interface and removes sync-specific types
redis/event.py Fixes async lock usage and adds async event types
redis/background.py Adds async recurring task scheduling capability
redis/asyncio/multidb/ Complete async implementation of multidb functionality

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Collaborator

@petyaslavova petyaslavova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM. You just need to fix the minor formatting issues reported by Copilot and it will be ready for merging.

vladvildanov and others added 2 commits September 10, 2025 10:23
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@vladvildanov vladvildanov merged commit ec8113b into feat/active-active Sep 10, 2025
1 check passed
@vladvildanov vladvildanov deleted the vv-async-multidb branch September 10, 2025 07:24
vladvildanov added a commit that referenced this pull request Oct 7, 2025
* MultiDbClient implementation (#3696)

* Added Database, Healthcheck, CircuitBreaker, FailureDetector

* Added DatabaseSelector, exceptions, refactored existing entities

* Added MultiDbConfig

* Added DatabaseConfig

* Added DatabaseConfig test coverage

* Renamed DatabaseSelector into FailoverStrategy

* Added CommandExecutor

* Updated healthcheck to close circuit on success

* Added thread-safeness

* Added missing thread-safeness

* Added missing thread-safenes for dispatcher

* Refactored client to keep databases in WeightedList

* Added database CRUD operations

* Added on-fly configuration

* Added background health checks

* Added background healthcheck + half-open event

* Refactored background scheduling

* Refactored healthchecks

* Removed code repetitions, fixed weight assignment, added loops enhancement, fixed data structure

* Refactored configuration

* Refactored failure detector

* Refactored retry logic

* Added scenario tests

* Added pybreaker optional dependency

* Added pybreaker to dev dependencies

* Rename tests directory

* Remove redundant checks

* Handle retries if default is not set

* Removed all Sentinel related

* Added support for Pipeline and transactions (#3707)

* Added Database, Healthcheck, CircuitBreaker, FailureDetector

* Added DatabaseSelector, exceptions, refactored existing entities

* Added MultiDbConfig

* Added DatabaseConfig

* Added DatabaseConfig test coverage

* Renamed DatabaseSelector into FailoverStrategy

* Added CommandExecutor

* Updated healthcheck to close circuit on success

* Added thread-safeness

* Added missing thread-safeness

* Added missing thread-safenes for dispatcher

* Refactored client to keep databases in WeightedList

* Added database CRUD operations

* Added on-fly configuration

* Added background health checks

* Added background healthcheck + half-open event

* Refactored background scheduling

* Added support for Active-Active pipeline

* Refactored healthchecks

* Added Pipeline testing

* Added support for transactions

* Removed code repetitions, fixed weight assignment, added loops enhancement, fixed data structure

* Added missing doc blocks

* Refactored configuration

* Refactored failure detector

* Refactored retry logic

* Added scenario tests

* Added pybreaker optional dependency

* Added pybreaker to dev dependencies

* Rename tests directory

* Added scenario tests for Pipeline and Transaction

* Added handling of ConnectionRefusedError, added timeouts so cluster could recover

* Increased timeouts

* Refactored integration tests

* Fixed property name

* Removed sentinels

* Removed unused method

* Added support for Pub/Sub mode in MultiDbClient (#3722)

* Added Database, Healthcheck, CircuitBreaker, FailureDetector

* Added DatabaseSelector, exceptions, refactored existing entities

* Added MultiDbConfig

* Added DatabaseConfig

* Added DatabaseConfig test coverage

* Renamed DatabaseSelector into FailoverStrategy

* Added CommandExecutor

* Updated healthcheck to close circuit on success

* Added thread-safeness

* Added missing thread-safeness

* Added missing thread-safenes for dispatcher

* Refactored client to keep databases in WeightedList

* Added database CRUD operations

* Added on-fly configuration

* Added background health checks

* Added background healthcheck + half-open event

* Refactored background scheduling

* Added support for Active-Active pipeline

* Refactored healthchecks

* Added Pipeline testing

* Added support for transactions

* Removed code repetitions, fixed weight assignment, added loops enhancement, fixed data structure

* Added missing doc blocks

* Added support for Pub/Sub in MultiDBClient

* Refactored configuration

* Refactored failure detector

* Refactored retry logic

* Added scenario tests

* Added pybreaker optional dependency

* Added pybreaker to dev dependencies

* Rename tests directory

* Added scenario tests for Pipeline and Transaction

* Added handling of ConnectionRefusedError, added timeouts so cluster could recover

* Increased timeouts

* Refactored integration tests

* Added scenario tests for Pub/Sub

* Updated healthcheck retry

* Increased timeout to avoid unprepared state before tests

* Added backoff retry and changed timeouts

* Added retry for healthchecks to avoid fluctuations

* Changed retry configuration for healthchecks

* Fixed property name

* Added check for thread results

* Refactored docblocks (#3744)

* Refactored healthcheck and failure detector to extend default one (#3747)

* Added MultiDbClient support with OSS Cluster API (#3734)

* Added Database, Healthcheck, CircuitBreaker, FailureDetector

* Added DatabaseSelector, exceptions, refactored existing entities

* Added MultiDbConfig

* Added DatabaseConfig

* Added DatabaseConfig test coverage

* Renamed DatabaseSelector into FailoverStrategy

* Added CommandExecutor

* Updated healthcheck to close circuit on success

* Added thread-safeness

* Added missing thread-safeness

* Added missing thread-safenes for dispatcher

* Refactored client to keep databases in WeightedList

* Added database CRUD operations

* Added on-fly configuration

* Added background health checks

* Added background healthcheck + half-open event

* Refactored background scheduling

* Added support for Active-Active pipeline

* Refactored healthchecks

* Added Pipeline testing

* Added support for transactions

* Removed code repetitions, fixed weight assignment, added loops enhancement, fixed data structure

* Added missing doc blocks

* Added support for Pub/Sub in MultiDBClient

* Refactored configuration

* Refactored failure detector

* Refactored retry logic

* Added scenario tests

* Added pybreaker optional dependency

* Added pybreaker to dev dependencies

* Rename tests directory

* Added scenario tests for Pipeline and Transaction

* Added handling of ConnectionRefusedError, added timeouts so cluster could recover

* Increased timeouts

* Refactored integration tests

* Added scenario tests for Pub/Sub

* Updated healthcheck retry

* Increased timeout to avoid unprepared state before tests

* Added backoff retry and changed timeouts

* Added retry for healthchecks to avoid fluctuations

* Changed retry configuration for healthchecks

* Fixed property name

* Added check for thread results

* Added MultiDbClient support with OSS Cluster API

* Removed database statuses

* Increased test timeouts

* Increased retry timeout

* Increased timeout retries

* Updated base threshold for retries

* Fixed flacky tests

* Added missing docblocks

* Added LagAwareHealthCheck for MultiDBClient (#3737)

* Added LagAwareHealthcheck

* Added testing for LagAwareHealthCheck

* Fixed timeouts

* Added lag tollerance parameter

* Decreased messages_count due to increased timeouts

* Added docblocks

* Added missing type hints

* Fixed url

* Refactored tests, URL and cluster support

* Use primary node to send an API request

* Added comment about RE bug

* Moved None type to the beginning

* Added health_check_url property to Database class

* Added lag_aware_tolerance parameter to LagAwareHealthcheck (#3752)

* Extract additional interfaces and abstract classes (#3754)

* Added async implementation of MultiDBClient (#3762)

* Extract additional interfaces and abstract classes

* Added base async components

* Added command executor

* Added recurring background tasks with event loop only

* Added MultiDBClient

* Added scenario and config tests

* Update redis/asyncio/multidb/healthcheck.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/test_asyncio/test_scenario/test_active_active.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Added pipeline and transaction support for MultiDBClient (#3763)

* Extract additional interfaces and abstract classes

* Added base async components

* Added command executor

* Added recurring background tasks with event loop only

* Added MultiDBClient

* Added scenario and config tests

* Added pipeline and transaction support for MultiDBClient

* Updated scenario tests to check failover

* Added pub/sub support for MultiDBClient (#3764)

* Extract additional interfaces and abstract classes

* Added base async components

* Added command executor

* Added recurring background tasks with event loop only

* Added MultiDBClient

* Added scenario and config tests

* Added pipeline and transaction support for MultiDBClient

* Added pub/sub support for MultiDBClient

* Added check for couroutines methods for pub/sub

* Added support for Lag-Aware Healthcheck and OSS Cluster API (#3768)

* Extract additional interfaces and abstract classes

* Added base async components

* Added command executor

* Added recurring background tasks with event loop only

* Added MultiDBClient

* Added scenario and config tests

* Added pipeline and transaction support for MultiDBClient

* Added pub/sub support for MultiDBClient

* Added check for couroutines methods for pub/sub

* Added OSS Cluster API support for MultiDBCLient

* Added support for Lag-Aware Healthcheck and OSS Cluster API

* Increased timeouts between tests

* Fixed space

* Refactored Healthcheck and Failover strategy logic (#3771)

* Extract additional interfaces and abstract classes

* Added base async components

* Added command executor

* Added recurring background tasks with event loop only

* Added MultiDBClient

* Added scenario and config tests

* Added pipeline and transaction support for MultiDBClient

* Added pub/sub support for MultiDBClient

* Added check for couroutines methods for pub/sub

* Added OSS Cluster API support for MultiDBCLient

* Added support for Lag-Aware Healthcheck and OSS Cluster API

* Increased timeouts between tests

* [Sync] Refactored healthcheck

* [Async] Refactored healthcheck

* [Sync] Refactored Failover Strategy

* [Async] Refactored Failover Strategy

* Changed default values according to a design doc

* [Async] Added Strategy Executor

* [Sync] Added Strategy Executor

* Apply comments

* Removed redundant dependency

* Fixed async tests

* Increased lag-aware tolerance

* Fixed typing issue, increase health_check_interval, added timeout handling

* Decreased retry cap, increased failure delay

* Fixed async teardown

* Fixed tests

* Added graceful connection closing, added graceful hc tasks termination

* Make sure active connection will be disconnected on failover

* Close cluster connection on failover

* Refactored Failure Detector (#3775)

* Decreased timeouts

* Added missing fixture

* Fixed None exception

* Codestyle changes

* Codestyle changes

* Skip async scenario tests

* Codestyle change

* Fixed unused arguments

* Refactored bg scheduler

* Fixed tests

* Fixed tests

* Codestyle fixes

* Reduce timeouts to avoid overlaping with healthcheck

* Marked tests non-clsuter only

* Update timeouts

* Skip scenario tests

* Updated timeouts

* Increased timeout

* Refactored tests

* Codestyle changes

* Added documentation for Active-Active (#3753)

* Added Active-Active documentation page

* Added documentation for Active-Active

* Refactored docs

* Refactored pipeline and transaction section

* Updated docs

* Extended list of words

* Re-write documentation

* Fixed spelling

* Update docs/multi_database.rst

Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>

* Apply suggested comments

* Fixed spelling

* Update docs/multi_database.rst

Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>

* Update docs/multi_database.rst

Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>

* Update docs/multi_database.rst

Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>

---------

Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>

* Refactor unstable tests

* Marked tests as non-clustered

* Codestyle changes

* Skipped tests in validating workflow

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: petyaslavova <petya.slavova@redis.com>
Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants