Performance regression: Datastore queries slower in 2.21.0 vs 2.20.1

[Nathanmalnoury]

Environment details

• OS type and version: Ubuntu 22.04 from google-22-full/python312 base image
• Python version: 3.12
• pip version: using google.python.pip 0.9.2 from buildpack
• google-cloud-datastore version: 2.21.0
• Other relevant packages:

  grpcio==1.76.0
  grpcio-status==1.71.2
  protobuf==5.29.5
  

Description of the Problem

After upgrading google-cloud-datastore from 2.20.1 to 2.21.0, Datastore queries became significantly slower.

In my production workload using NDB, a query that normally takes about 6 seconds with version 2.20.1 now takes around 20 seconds with 2.21.0.

Even when bypassing NDB and using the Datastore client directly, there is still a consistent slowdown.
Running the same query 15 times:

• 2.20.1: ~6.06 seconds (average)
• 2.21.0: ~11.03 seconds (average)

No other dependencies in the project changed (I've attach my full requirements before and after update below).

Code example

            client = datastore.Client()
            query = client.query(kind="Enterprise") # This kind has many entities and larger payloads
            results = list(query.fetch()) 

Requirements

new-requirements.txt (using 2.21.0)

old-requirements.txt (using 2.20.1)

(These are the full requirements, and involve a lot more than datastore)