Fix error when retaining policy_tags with access controls in replace mode #105
Conversation
…lace mode Refactored the retention logic for descriptions and policy_tags to handle the case where policy_tags have access controls, which was causing errors when retain_column_policy_tags was enabled. - Moved retention logic from buildSchema to post-table-creation update - buildSchema now only defines field structure, temporarily storing retained info - Added updateTableIfNeed to update table after creation when needed - column_options description takes precedence over retained description 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
| bigquery.update( | ||
| TableInfo.newBuilder( | ||
| table.getTableId(), | ||
| StandardTableDefinition.newBuilder().setSchema(patchSchema).build()) | ||
| .build()); |
There was a problem hiding this comment.
Could we have an error handling for bigquery.update ?
There was a problem hiding this comment.
@NamedPython
Thank you for the review.
Could you specify the type of exception handling you’re considering?
I believe it would fall under one of the following cases — could you please confirm?
-
Output an error log and throw a BigqueryException.
-
Some API errors do not affect the overall process, so we will ignore those errors in such cases.
-
Output a warning log and continue the process.
-
No change needed — just let the error thrown by bigquery.update propagate as is.
For reference, in the cases of getDataset, createDataset, and deleteTableOrPartition, neither the caller nor the callee performs any specific exception handling.
Summary
Background
When
retain_column_policy_tagswas enabled in replace mode and the existing table had policy_tags with accesscontrols, the operation would fail. This was because trying to set policy_tags with access controls during table
creation is not allowed by BigQuery.
Changes
buildSchemato a post-creation update processupdateTableIfNeedmethod to update table schema after creationstoreCachedSrcFieldsIfNeedto temporarily store source fields