Skip to content

Source BigQuery: allow to be in a different project #10311

Open
@kattos-aws

Description

Tell us about the problem you're trying to solve

I am trying to ingest data from a publicly available dataset on BigQuery (namely the bitcoin dataset).
This dataset is located in a project I am not owner of.

When I setup the BigQuery source connector and run it, I get the following error:

com.google.cloud.bigquery.BigQueryException: Access Denied: Project bigquery-public-data: User does
 not have bigquery.jobs.create permission in project bigquery-public-data.

I am able to query the dataset through the GCP console.

Describe the solution you’d like

I would like to be able to ingest the same through Airbyte.

Describe the alternative you’ve considered or used

None so far, suggestions would be welcome. I could have a job replicating the data into a project I own, but that is rather inconvenient, and incurs costs especially if the dataset is big.

Additional context

N/A

Are you willing to submit a PR?

Sorry, I can't commit to this due to lack of time.

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions