`listOperationsAsync` Type Mismatch Due to gax-nodejs Pagination Handling

[RyuSA]

Environment details

• Programming language: TypeScript
• OS: (Please specify your OS - e.g., macOS 13.6, Windows 11, Ubuntu 22.04) I used macOS, but it probably doesn't matter.
• Language runtime version: Node.js (Please specify your Node.js version - e.g., v18.17.0, v20.5.0) I used v22.12.0
• Package version:
  • "@google-cloud/discoveryengine": "^1.14.0"
  • "@google-cloud/translate": "^8.5.1"
  • "google-gax": "^4.4.1"

Steps to reproduce

1. Execute a Batch Job: Run a batch job using the Translation API (or any other API with long-running operations).
   Running the job multiple times can make the issue easier to observe.

Use the following curl command, replacing placeholders with your actual values:

curl -X POST \
    -H "Authorization: Bearer $(gcloud auth print-access-token)" \
    -H "Content-Type: application/json" \
    "https://translation.googleapis.com/v3/projects/${PROJECT_ID}/locations/${LOCATION}:batchTranslateText" \
    -d '{
    "sourceLanguageCode": "en",
    "targetLanguageCodes": ["es", "fr"],
    "inputConfigs": [
      {
        "gcsSource": {
          "inputUri": "gs://BUTCKET/soma/data"
        }
      }
    ],
    "outputConfig": {
        "gcsDestination": {
          "outputUriPrefix": "gs://BUTCKET/"
        }
      }
  }'

• ${PROJECT_ID}: Your Google Cloud Project ID.
• ${LOCATION}: The location (e.g., us-central1).
• gs://BUTCKET/...: Valid Cloud Storage paths.

2. Set up a TypeScript Environment: Install the necessary packages:

npm install @google-cloud/discoveryengine @google-cloud/translate
npm install -D @types/node ts-node typescript

3. Create and Run TypeScript Code: Create a TypeScript file (e.g., index.ts) with the following code.

import { protos } from '@google-cloud/translate';
import { TranslationServiceClient } from '@google-cloud/translate/build/src/v3';

async function main() {
    const client = new TranslationServiceClient({
        projectId: projectId
    });
    const request = new protos.google.longrunning.ListOperationsRequest({
        name: "projects/PROJECT_ID/locations/LOCATION" // Replace project/location
    });
    const iterable = client.listOperationsAsync(request);
    for await (const raw of iterable) {
        console.log(raw.operations);
        // console.log(raw.done); // Property 'done' does not exist on type 'ListOperationsResponse'.ts
    }
}
main().catch(err => {
    console.error(err);
    process.exit(1);
});

Then, run the code: npx ts-node index.ts

4. Observe the Results:

   • The code will compile without errors.
     • But the output will be undefined.
   • If you uncomment console.log(raw.done), you'll get a TypeScript compilation error: Property 'done' does not exist on type 'ListOperationsResponse'.ts.
     • Inspecting raw in a debugger suggests it's an IOperation, not a ListOperationsResponse.

Detailed Explanation and Root Cause:

The core issue is a type mismatch between the declared return type of listOperationsAsync and the actual value it returns. This is due to how gax-nodejs handles pagination for long-running operations.

This SDK utilizes gax-nodejs, which provides automatic pagination. Let's examine the code step-by-step.

operationsClient.listOperationsAsync uses innerApiCalls.listOperations within descriptor.listOperations.asyncIterate. Notably, the return value is cast to AsyncIterable<protos.google.longrunning.ListOperationsResponse>:
https://github.com/googleapis/gax-nodejs/blob/main/gax/src/operationsClient.ts#L444

descriptor.listOperations is defined here: https://github.com/googleapis/gax-nodejs/blob/main/gax/src/operationsClient.ts#L109. It passes the key for nextPageToken to PageDescriptor, enabling automatic pagination.

Looking at the internal implementation, the result obtained from apiCall is pushed to a cache, and an iterator that can access this cache is returned. This result should be the data unpacked from protos.google.longrunning.ListOperationsResponse:
https://github.com/googleapis/gax-nodejs/blob/main/gax/src/paginationCalls/pageDescriptor.ts#L190

Referring to the documentation, the only field that can populate the array result in google.longrunning.ListOperationsResponse is google.longrunning.IOperation[]. (This link is for the Speech API, but the structure should be the same for other services):
https://cloud.google.com/nodejs/docs/reference/speech/latest/speech/protos.google.longrunning.listoperationsresponse

Working backward, operationsClient.listOperationsAsync should receive AsyncIterable<protos.google.longrunning.IOperation> and not AsyncIterable<protos.google.longrunning.ListOperationsResponse>.

When actually running the code, console.log(raw.done) outputs "true", which confirms my theory.

Proposed Solution:

We can cast the type cast at https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/src/v1/document_service_client.ts#L202 to AsyncIterable<protos.google.longrunning.IOperation>, like this.

  listOperationsAsync(
    request: protos.google.longrunning.ListOperationsRequest,
    options?: gax.CallOptions
-  ): AsyncIterable<protos.google.longrunning.ListOperationsResponse> {
+  ): AsyncIterable<protos.google.longrunning.IOperation> {
    options = options || {};
    options.otherArgs = options.otherArgs || {};
    options.otherArgs.headers = options.otherArgs.headers || {};
    options.otherArgs.headers['x-goog-request-params'] =
      this._gaxModule.routingHeader.fromParams({
        name: request.name ?? '',
      });
-    return this.operationsClient.listOperationsAsync(request, options);
+    return this.operationsClient.listOperationsAsync(request, options) as AsyncIterable<protos.google.longrunning.IOperation>;
  }

But I'm not sure how these code are generated...

Workaround

        const iterable = client.listOperationsAsync(request);
        for await (const raw of iterable) {
            const data = raw as protos.google.longrunning.IOperation;
            // do something
        }

Impact and Concerns:

This solution is a potentially breaking change for other Google Cloud SDKs that depend on gax-nodejs. We need to carefully apply the solution.