Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve documentation: It's unclear how to conenct to a local emulator #3128

Open
etrandafir93 opened this issue Feb 9, 2024 · 3 comments
Open
Assignees
Labels
api: bigquery Issues related to the googleapis/java-bigquery API.

Comments

@etrandafir93
Copy link

etrandafir93 commented Feb 9, 2024

We are testing the interaction with BigQuery using Testcontainers and their glcoud module.
Internally, this will use the BigQuery emulator from goccy.

Although everything is ok for the REST Api and we can test everything related to it using this setup, setting up a JsonStreamWriter that connects to the emulator via grpc it's pretty hard to achieve. (Same question was asked in this StackOverflow post)

I couldn't find anything related to this in the official documentation or in the Javadocs. What we need is to understand how to configure the Java client to point to the container started locally.

I have tried a few different configurations like this:

JsonStreamWriter writer = JsonStreamWriter.newBuilder(
    TableName.of(BQ_DATASET, BQ_PROJECT, BQ_TABLE).toString(),
    BigQueryWriteClient.create(
        BigQueryWriteSettings.newBuilder()
            .setTransportChannelProvider(
                FixedTransportChannelProvider.create(
                    GrpcTransportChannel.create(
                        NettyChannelBuilder.forTarget("localhost:9060")
                            .usePlaintext()
                            .build()
                    )
                )
            )
            .setCredentialsProvider(NoCredentialsProvider.create())
            .build()
    )
).build();

Would it be possible to update the docs or the GitHub repo and provide an example of how can this be achieved?

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/java-bigquery API. label Feb 9, 2024
@etrandafir93 etrandafir93 changed the title Improve documentation: It's unclear how to conenct to an emulator Improve documentation: It's unclear how to conenct to a local emulator Feb 16, 2024
@PhongChuong
Copy link
Contributor

PhongChuong commented Mar 7, 2024

Hi @etrandafir93 , testcontainers is not part of googleapis libraries. As such, providing an example on how to integrate with it would be difficult for us as we do not know their product. I would suggest asking the owners of Testcontainers directly as they will have more experience with integrating the two.

I have not tried this myself but another option would be to look into setEndpoint() in BigQueryWriteSettings when creating BigQueryWriteClient for the JsonStreamWriter.

@etrandafir93
Copy link
Author

hello @PhongChuong - thanks for reaching out!

To summarize, I'm asking if the documentation/javadocs for this project (the java-bigquey client library) can be improved a bit so that it's clear how to configure it to point to a different grpc endpoint. Since the library supports this feature, it'd be helpful to have a clearer example/docs to guide users in using it.

I did mention testcontainers here just to offer a bit of context about what I'm trying to achieve, but it's not highly relevant. My main aim is to use the java lib to connect to a different grpc endpoint.

I have tried multiple configurations, including BigQueryWriteSettings::setEndpoint, as illustrated in the StackOverflow question, but none of them worked.

@PhongChuong
Copy link
Contributor

Thanks for the clarification.
We'll keep this open until we can find a tech writer who can work on this item.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/java-bigquery API.
Projects
None yet
Development

No branches or pull requests

3 participants