feat: Introduce DataFormatOptions to configure the output of BigQuery data types #4010

lqiu96 · 2025-12-03T22:16:37Z

See b/447623336 for more information.

Introduces a DataFormatOptions class to configure how to format the data for BigQuery outputs. It includes the existing functionality for useInt64Timestamps that is currently a setting in BigQueryOptions. Those settings are absorbed into the new DataFormatOptions class.

This PR does not change the logic for the existing BigQuery code, but converts any existing use cases of BigQueryOptions's useInt64Timestamps to now reference BigQueryOption's DataFormatOptions.useInt64Timestamps value instead.

…tOption's variant

gemini-code-assist · 2025-12-03T22:17:03Z

Summary of Changes

Hello @lqiu96, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors how data formatting options are managed within the BigQuery client library by introducing a new DataFormatOptions wrapper class. This change centralizes configuration for aspects like useInt64Timestamps, deprecating older direct setters and getters in BigQueryOptions in favor of the new, more extensible object. The update aims to improve the clarity and future-proofing of data format settings across the library.

Highlights

New DataFormatOptions Class: A dedicated DataFormatOptions class has been introduced to encapsulate data formatting configurations, including timestamp handling, providing a centralized and extensible structure.
Refactored BigQueryOptions: The BigQueryOptions class now utilizes the new DataFormatOptions object to manage data format settings, replacing direct boolean flags for improved modularity.
Deprecation of Direct Timestamp Flags: The setUseInt64Timestamps and getUseInt64Timestamps methods in BigQueryOptions have been marked as @ObsoleteApi, guiding users to adopt the new DataFormatOptions wrapper for timestamp configuration.
Updated Query Request Handling: The QueryRequestInfo constructor has been modified to accept the DataFormatOptions object, ensuring consistent application of formatting preferences during query execution.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces a DataFormatOptions wrapper to encapsulate data format settings, deprecating the old useInt64Timestamps option. The changes are well-structured, but I've found a couple of issues. There's a potential for NullPointerException in the new DataFormatOptions class, and a state inconsistency issue in BigQueryOptions due to the new obsolete setter. My review includes suggestions to fix these problems.

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/DataFormatOptions.java

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryOptions.java

lqiu96 · 2025-12-03T23:46:28Z

/gemini review

gemini-code-assist

Code Review

The pull request introduces a new DataFormatOptions class to centralize data format configurations, which is a great improvement for configurability. The changes correctly deprecate the old useInt64Timestamps option in BigQueryOptions and provide a backward-compatible way to handle it. However, the refactoring appears to be incomplete. While QueryRequestInfo is updated, several other places in BigQueryImpl still call the obsolete getOptions().getUseInt64Timestamps(). These should be updated to use the new DataFormatOptions to ensure consistent behavior. I've also added a few specific comments on the new DataFormatOptions class and on improving Javadoc clarity for the deprecated methods.

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/DataFormatOptions.java

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryOptions.java

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/DataFormatOptions.java

lqiu96 · 2025-12-04T23:53:25Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces a new DataFormatOptions class to encapsulate data formatting settings, starting with useInt64Timestamp. The changes are well-implemented across the codebase, with good handling of backward compatibility by deprecating old methods and ensuring the new and old settings work together correctly. New tests have been added to verify this behavior. I've found one potential issue in the new DataFormatOptions class that could lead to a runtime exception, for which I've provided a suggestion.

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/DataFormatOptions.java

jinseopkim0 · 2025-12-05T19:39:54Z

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryOptions.java

+     *
+     * <p>{@code DataFormatOptions.newBuilder().setUseInt64Timestamp(...).build()}
+     */
+    @ObsoleteApi("Use setDataFormatOptions(DataFormatOptions) instead")


Is there a reason you would prefer @ObsoleteApi over @Deprecated?

@Deprecated could result downstream customer's CI jobs failing based on their compiler settings. We prefer @ObsoleteApi as a first warning and then moving to @Deprecated in a future major version.

jinseopkim0 · 2025-12-05T19:42:44Z

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/DataFormatOptions.java

+    }
+    if (request.getTimestampOutputFormat() != null) {
+      builder.timestampFormatOptions(
+          TimestampFormatOptions.valueOf(request.getTimestampOutputFormat()));


If BQ backend adds a new format, this would fail, right? Can we try-catch, so that we would fall back to TIMESTAMP_OUTPUT_FORMAT_UNSPECIFIED?

Ah good point. This format was created from the proto files, but really only customers should be creating and setting it on the client side. This shouldn't come back as a server response at all.

I'll remove the fromPb() method.

… data types (#4010) * feat: Create DataFormatOptions in BigQuery * feat: Add Builder class for DataFormatOptions * fix: Update existing references of useInt64Timestamp to use DataFormatOption's variant * chore: Fix lint issues * chore: Address PR feedback * chore: Add tests for useInt64Timestamp behavior * chore: Address failing tests and GCA * chore: Remove unused fromPb method

@gemini-code-assist

* chore: sync with last release from main branch * feat: Introduce DataFormatOptions to configure the output of BigQuery data types (#4010) * feat: Create DataFormatOptions in BigQuery * feat: Add Builder class for DataFormatOptions * fix: Update existing references of useInt64Timestamp to use DataFormatOption's variant * chore: Fix lint issues * chore: Address PR feedback * chore: Add tests for useInt64Timestamp behavior * chore: Address failing tests and GCA * chore: Remove unused fromPb method * feat: Add timestamp_precision to Field (#4014) * feat: Add timestamp_precision to Field * Apply suggestion from @gemini-code-assist[bot] Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * chore: Address GCA PR feedback * chore: Fix typo * chore: Remove default value * chore: Address PR feedback --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * chore: Use custom timestamp validator for ISO8601 timestamps with more than nanosecond precision (#4017) * chore: Use custom timestamp validator for ISO8601 timestamps with more than nanosecond precision * chore: Rename helper method to validateTimestamp * chore: Address GCA comments * chore: Address last GCA comment * chore: Update to use assert helper method * test: Clean up resources created in ITs (#4024) * chore: Cleanup initialized resources * chore: Delete created dataset * chore: Use try-with to close stream * chore: Cleanup Intellij test warnings (#4026) * chore: Cleanup initialized resources * chore: Use try-with to close stream * chore: Cleanup Intellij test warnings * chore: Allow for floating point inaccuracies * chore: Add otel delete operation * deps: update actions/upload-artifact action to v6 (#4027) Co-authored-by: Blake Li <blakeli@google.com> * chore: Use assertThrows in tests (#4028) * chore: Cleanup initialized resources * chore: Use try-with to close stream * chore: Cleanup Intellij test warnings * chore: Allow for floating point inaccuracies * chore: Add otel delete operation * chore: Clean up rest of the IT test file * chore: Add GCA feedback * chore: Update renovate.json (#4031) deps update from bigquerystorage was incorrectly marked as chore(deps). This would cause the upgrade not show up in the release notes and also not trigger release please. This is because the pattern /^com.google.cloud:google-cloud-bigquery is configured in renovate.json, which matches both bigquery and bigquerystorage. However, I believe the original intention is only for bigquery update in samples, not for bigquerystorage update. Fixing the pattern so that the String must ends with bigquery * chore(main): release 2.57.2-SNAPSHOT (#4032) Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> * chore: fix unresolved conflicts --------- Co-authored-by: Lawrence Qiu <lawrenceqiu@google.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Mend Renovate <bot@renovateapp.com> Co-authored-by: Blake Li <blakeli@google.com> Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

lqiu96 added 3 commits December 3, 2025 14:05

feat: Create DataFormatOptions in BigQuery

7c8c47f

feat: Add Builder class for DataFormatOptions

9caca67

fix: Update existing references of useInt64Timestamp to use DataForma…

6e16c81

…tOption's variant

lqiu96 requested a review from jinseopkim0 December 3, 2025 22:16

product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/java-bigquery API. labels Dec 3, 2025

lqiu96 changed the title ~~Dataformatoptions wrapper~~ feat: Introduce DataFormatOptions to configure the output of BigQuery data types Dec 3, 2025

gemini-code-assist bot reviewed Dec 3, 2025

View reviewed changes

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/DataFormatOptions.java Outdated Show resolved Hide resolved

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/BigQueryOptions.java Show resolved Hide resolved

chore: Fix lint issues

cfc0b26

gemini-code-assist bot reviewed Dec 3, 2025

View reviewed changes

lqiu96 added 2 commits December 3, 2025 19:08

chore: Address PR feedback

dd4e58f

chore: Add tests for useInt64Timestamp behavior

65e0927

product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Dec 4, 2025

Merge branch 'main' into dataformatoptions-wrapper

7952544

gemini-code-assist bot reviewed Dec 4, 2025

View reviewed changes

google-cloud-bigquery/src/main/java/com/google/cloud/bigquery/DataFormatOptions.java Outdated Show resolved Hide resolved

chore: Address failing tests and GCA

ad4128c

lqiu96 marked this pull request as ready for review December 5, 2025 00:05

lqiu96 requested review from a team as code owners December 5, 2025 00:05

jinseopkim0 requested changes Dec 5, 2025

View reviewed changes

chore: Remove unused fromPb method

0affa6d

product-auto-label bot added size: m Pull request size is medium. and removed size: l Pull request size is large. labels Dec 5, 2025

lqiu96 requested a review from jinseopkim0 December 5, 2025 20:37

lqiu96 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 5, 2025

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Dec 5, 2025

jinseopkim0 approved these changes Dec 6, 2025

View reviewed changes

lqiu96 merged commit 6dcc900 into main Dec 8, 2025
24 checks passed

lqiu96 deleted the dataformatoptions-wrapper branch December 8, 2025 15:32

release-please bot mentioned this pull request Dec 9, 2025

chore(main): release 2.57.0 #4021

Merged

feat: Introduce DataFormatOptions to configure the output of BigQuery data types #4010

feat: Introduce DataFormatOptions to configure the output of BigQuery data types #4010

Uh oh!

Conversation

lqiu96 commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist bot commented Dec 3, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

lqiu96 commented Dec 3, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lqiu96 commented Dec 4, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

jinseopkim0 Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

lqiu96 Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

jinseopkim0 Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

lqiu96 Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lqiu96 commented Dec 3, 2025 •

edited

Loading