Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add excluded locations on client and request levels #40022

Open
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

allenkim0129
Copy link
Contributor

@allenkim0129 allenkim0129 commented Mar 11, 2025

This change will add excluded locations on client and request levels.

Design doc: https://microsoft-my.sharepoint.com/:w:/p/allekim/EWf4tc65_f9GiOv9aD2XkLMBD9h2ZAAXCcFTKghjutvqcg?e=AlD8w1

On Client level:

def excluded_locations_client_level_sample():
    preferred_locations = ['West US 3', 'West US', 'East US 2']
    excluded_locations = ['West US 3', 'West US']
    client = CosmosClient(
                HOST,
                MASTER_KEY,
                preferred_locations=preferred_locations,
                excluded_locations=excluded_locations
    )

    db = client.create_database(DATABASE_ID)
    container = db.create_container(id=CONTAINER_ID, partition_key=PARTITION_KEY)

    # For write operations with single master account, write endpoint will be the default endpoint,
    # since preferred_locations or excluded_locations are ignored and used
    container.create_item(get_test_item(0))

    # For read operations, read endpoints will be 'preferred_locations' - 'excluded_locations'.
    # In our sample, ['West US 3', 'West US', 'East US 2'] - ['West US 3', 'West US'] => ['East US 2'],
    # therefore 'East US 2' will be the read endpoint, and items will be read from 'East US 2' location
    item = container.read_item(item='Item_0', partition_key='Item_0')
  • Note: During client creation, excluded locations can be set by using the excluded_locations parameter. This set will filter any existing preferred locations. If excluded locations filtered all preferred locations, Cosmos DB will use the default endpoints.

On Request level:

def excluded_locations_request_level_sample():
    preferred_locations = ['West US 3', 'West US', 'East US 2']
    excluded_locations_on_client = ['West US 3', 'West US']
    excluded_locations_on_request = ['West US 3']
    client = CosmosClient(
                HOST,
                MASTER_KEY,
                preferred_locations=preferred_locations,
                excluded_locations=excluded_locations_on_client
    )

    db = client.create_database(DATABASE_ID)
    container = db.create_container(id=CONTAINER_ID, partition_key=PARTITION_KEY)

    # For write operations with single master account, write endpoint will be the default endpoint,
    # since preferred_locations or excluded_locations are ignored and used
    container.create_item(get_test_item(0))

    # For read operations, read endpoints will be 'preferred_locations' - 'excluded_locations'.
    # However, in our sample, since the excluded_locations` were passed with the read request, the `excluded_location`
    # will be replaced with the locations from request, ['West US 3']. The `excluded_locations` on request always takes
    # the highest priority!
    # With the excluded_locations on request, the read endpoints will be ['West US', 'East US 2']
    #   ['West US 3', 'West US', 'East US 2'] - ['West US 3'] => ['West US', 'East US 2']
    # Therefore, items will be read from 'West US' or 'East US 2' location
    item = container.read_item(item='Item_0', partition_key='Item_0', excluded_locations=excluded_locations_on_request)
  • Note: The excluded locations on requests level can be used by passing excluded_locations parameters along with any other parameters for Container level APIs. This excluded_location parameter will override the existing exlcuded locations on client level. This means also means that if a user can manupilate the client level excluded locations for the specific request. Moreover, if a user wants to skip all client level excluded locations, the user can simply pass an empty list [] of excluded locations to skip all excluded location filters.

Client level API works with excluded_location:

  • read_item
  • read_all_items
  • query_items_change_feed
  • query_items
  • replace_item
  • upsert_item
  • upsert_item
  • create_item
  • patch_item
  • execute_item_batch
  • delete_item
  • delete_all_items_by_partition_key

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

@allenkim0129 allenkim0129 changed the title Users/allekim/feature/add exclusive region Add excluded locations on client and request levels Mar 11, 2025
@azure-sdk
Copy link
Collaborator

API change check

APIView has identified API level changes in this PR and created following API reviews.

azure-cosmos

@allenkim0129 allenkim0129 marked this pull request as ready for review March 12, 2025 00:57
@Copilot Copilot bot review requested due to automatic review settings March 12, 2025 00:57
@allenkim0129 allenkim0129 requested review from annatisch and a team as code owners March 12, 2025 00:57
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request adds support for specifying excluded locations at both the client and request levels, ensuring that the Cosmos DB SDK can properly filter preferred locations when excluded regions are defined. Key changes include:

  • New sample functions and updated documentation in the samples module demonstrating how to set excluded locations.
  • Modifications throughout the client connection, request object, and location cache logic (both sync and async) to propagate and use the excluded_locations parameter.
  • Updates to tests and documentation to include the new excluded_locations parameter in configuration and API docstrings.

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

Show a summary per file
File Description
sdk/cosmos/azure-cosmos/samples/excluded_locations.py Adds new sample functions that demonstrate how to use excluded locations on both client and request levels.
sdk/cosmos/azure-cosmos/test/test_globaldb.py Adds a new test for verifying excluded locations behavior.
sdk/cosmos/azure-cosmos/test/test_location_cache.py Introduces parameterized tests for verifying regional endpoint filtering based on excluded locations.
azure/cosmos/_request_object.py Updates the RequestObject to set excluded locations from options.
azure/cosmos/_cosmos_client_connection(.py and aio version) Inserts calls to set excluded locations on request objects in various operations.
azure/cosmos/_location_cache.py Updates logic to filter read and write regional endpoints based on excluded locations and adds utility methods for this purpose.
azure/cosmos/cosmos_client.py Integrates the excluded_locations parameter into the connection policy used to initialize the client.
azure/cosmos/documents.py Updates documentation for ConnectionPolicy to include the new excluded_locations setting.
azure/cosmos/_base.py Adds the mapping for 'excluded_locations' in the common options used for building request headers.
azure/cosmos/aio/_global_endpoint_manager_async.py and azure/cosmos/_global_endpoint_manager.py Pass the connection_policy (including excluded_locations) to the LocationCache.
azure/cosmos/container.py Updates function docstrings to include the new excluded_locations parameter.
Comments suppressed due to low confidence (1)

sdk/cosmos/azure-cosmos/samples/excluded_locations.py:98

  • The word "manupilate" appears to be misspelled. Consider replacing it with "manipulate" for clarity.
if a user can manupilate the client level excluded locations for the specific request.

Copy link
Member

@simorenoh simorenoh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good overall, thanks Allen! Just had some questions.

Also, like Abhijeet mentioned, let's make sure to run some manual tests on this logic beforehand to make sure we don't miss anything.

@tvaron3
Copy link
Member

tvaron3 commented Mar 25, 2025

In the future, let's try not to squash commits. It makes it much harder to track down original prs and when changes were made.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

7 participants