[source-bigquery] sync fails with large table size

[gatsby003]

Connector Name

source-bigquery

Connector Version

0.4.2

What step the error happened?

During the sync

Relevant information

I am trying to sync data from bigquery to snowflake. While trying to sync a table of large size I am facing 403 forbidden error. This seems to be a common issue with bigquery syncs. Attaching relevant log outputs in below.

Relevant log output

Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
GET https://www.googleapis.com/bigquery/v2/projects/customer-data-408810/queries/177df063-ea20-44ef-b46c-8d0c5e6e4867?location=europe-west3&maxResults=0&prettyPrint=false
{
  "code": 403,
  "errors": [
    {
      "domain": "global",
      "message": "Response too large to return. Consider specifying a destination table in your job configuration. For more details, see https://cloud.google.com/bigquery/troubleshooting-errors",
      "reason": "responseTooLarge"
    }
  ],
  "message": "Response too large to return. Consider specifying a destination table in your job configuration. For more details, see https://cloud.google.com/bigquery/troubleshooting-errors",
  "status": "PERMISSION_DENIED"
}

Contribute

• Yes, I want to contribute