out_bigquery: Add debug log for response payload when HTTP 200 OK

[oz-rw]

This commit adds debug logging of the response payload. Even with HTTP 200 OK status response, BigQuery may return an error message in the payload. This can lead to data not being inserted into BigQuery silently.

See https://cloud.google.com/bigquery/docs/streaming-data-into-bigquery#streaming-success for reference:

Success HTTP response codes
Even if you receive a success HTTP response, you'll need to check the insertErrors property of the response to determine whether the row insertions were successful because it's possible that BigQuery was only partially successful at inserting the rows. You might encounter one of the following scenarios:
    
- All rows inserted successfully.
  If the insertErrors property is an empty list, all of the rows were inserted successfully.
- Some rows inserted successfully.
  Except in cases where there is a schema mismatch in any of the rows, rows indicated in the insertErrors property are not inserted, and all other rows are inserted successfully. The errors property contains detailed information about why each unsuccessful row failed. The index property indicates the 0-based row index of the request that the error applies to.
- None of the rows inserted successfully.
  If BigQuery encounters a schema mismatch on individual rows in the request, none of the rows are inserted and an insertErrors entry is returned for each row, even the rows that did not have a schema mismatch. Rows that did not have a schema mismatch have an error with the reason property set to stopped, and can be re-sent as-is. Rows that failed include detailed information about the schema mismatch. To learn about the supported protocol buffer types for each BigQuery data type, see [Data type conversions](https://cloud.google.com/bigquery/docs/write-api#data_type_conversions).

Now it is easier to detect such cases with debug logging:

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• [N/A] Example configuration file for the change
• Debug log output from testing the change

[2024/04/19 07:59:05] [debug] [output:bigquery:bigquery.1] HTTP Status=200
[2024/04/19 07:59:05] [debug] [output:bigquery:bigquery.1] response
{
  "kind": "bigquery#tableDataInsertAllResponse",
  "insertErrors": [
    {
      "index": 0,
      "errors": [
        {
          "reason": "invalid",
          "location": "verticaluncertainty",
          "debugInfo": "",
          "message": "Invalid NUMERIC value: 5233.56298828125"
        }
      ]
    }
  ]
}

• [N/A] Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• [N/A] Run local packaging test showing all targets (including any new ones) build.
• [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• [N/A] Documentation required for this feature

Backporting

• [N/A] Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[oz-rw]

Ping

[oz-rw]

@edsiper, any chance someone could take a look at my PR? or might give some feedback if anything is missing. Thanks!

[github-actions]

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[dayvar14]

Can this get reviewed? I'm having similar issues with rows failing to be inserted silently. This output would be very helpful in debugging.