Skip to content

GetItemLinqQueryable with _ts in the predicate returns stale (seems to be old snapshot) data #4230

Open

Description

Describe the bug
When I use GetItemLinqQueryable with _ts in the LINQ predicate, the SDK, or maybe the service itself, returns stale data that appears to be some old snapshot of the data instead of the document in its latest state.

    var query = cosmosClient.GetDatabase(Constants.COSMOS_DATABASE_NAME).GetContainer(Constants.COSMOS_CONTAINER_NAME)
            .GetItemLinqQueryable<T>(requestOptions: new QueryRequestOptions { PartitionKey = partitionKey })
            .Where(x => !x.IsDeleted)
            .Where(x => x.UpdatedEpoch <= 123456789);

    using (var feedIterator = query.ToFeedIterator())
    {
        while (feedIterator.HasMoreResults)
        {
            foreach (var entity in await feedIterator.ReadNextAsync().ConfigureAwait(false))
            {
                yield return entity;
            }
        }
    }

UpdatedEpoch is a property of my JSON document with the following JSON property attribute to map it to the _ts field that is included in every document.

  [JsonProperty("_ts")]
  public long UpdatedEpoch { get; set; }

When I use the predicate x => x.UpdatedEpoch <= 1703289600, I get version A (an old version) of the document back (NOT expected), but if I query by document id, I get the latest version, let's call it version B.

I have verified the following:

  1. A and B have the same id and partition key hence they are the same document
  2. A and B have different _ts, the _ts for A is indeed smaller than 1703289600 as I specified in my predicate and B (the latest version) has _ts larger than 1703289600
  3. From Azure Portal using Data Explorer to load the document by id I get Version B (Expected)
  4. From SDK using point read ReadItemAsync by id and partition key I get Version B (Expected)
  5. From SDK GetItemLinqQueryable using this predicate x => x.Id == "my_id" && x.UpdatedEpoch <= 1703289600 the query return nothing (Expected because the latest version _ts is greater than 1703289600
  6. From SDK GetItemLinqQueryable using this predicate x => x.Id == "my_id" and I get Version B (Expected)

Expected behavior
When I query with x => x.UpdatedEpoch <= 1703289600 I expected the query to return the latest version of all documents that were last updated before the given timestamp for the _ts field, not a snapshotted version of some document that is different from its latest state.

Actual behavior
I get an old snapshot of the document. Note that I do not keep versioning of my documents so when I read the document I expected to ever get 1 version back yet somehow Cosmos DB is returning an old version that it kept internally (seems to me that way).

My Cosmos DB is setup with Eventual Consistency level, 1 Write region and no Read region as shown below so I am not expecting any stale read to happen at all, even with Eventual Consistency.
image

I've tried to change my consistency level to Strong and that did not change the behavior of above query. I was still getting Version A (old) of the document back when I try to include other properties in my predicate, as long as it is not the id of the document.

Environment summary
SDK Version: Microsoft.Azure.Cosmos Version 3.37.0
OS Version Windows 21H2, 22000.2538

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions