Expose BigQuery scopes

[jlowin]

The default scopes for the BigQuery Client do not include Google Drive, which makes it difficult to access federated data sources in Google Sheets.

This can be worked around by manually constructing service account credentials with the appropriate scopes but that feels cumbersome. Am I not seeing a hook that allows scope modification when instantiating bigquery.Client()?

[daspecster]

I don't believe that Google Drive is directly support to import data from for BigQuery.

This is the list of supported sources that I'm aware of.
https://cloud.google.com/bigquery/loading-data

[dhermes]

@tswast Can you comment?

[tswast]

Google Drive is in fact a supported federated query source. https://cloud.google.com/bigquery/external-data-sources#creating_an_external_table_using_google_drive

I don't think it should in the default scopes, but we definitely need a way to amend the scopes used in the client. This will be an issue for any of the external data sources described in #2372 For example, Cloud Storage will need it's scope to query files there.

[jlowin]

@tswast I agree it shouldn't be in the default set, but it would be great to be able to flag it from the default Client. That way it could also be passed as a kwarg to bigquery.Client.from_service_account_json() without modification.

[jlowin]

@daspecster I think the reason it isn't listed there is because unlike GCS files, GDrive files (including Google Sheets) can only be external -- not loaded as native tables.

[clifton]

Ran into this today and couldn't find a work-around (even overwriting scopes wouldn't work!). In the mean-time I'm manually re-materializing the google sheets table.

[cdchan]

I had the same issue as well -- created a client subclass that added the Drive scope but still couldn't get it to work.

[tswast]

In the meantime you should be able to create a credentials object using google.auth.default(scopes=...)

https://google-auth.readthedocs.io/en/latest/reference/google.auth.html#google.auth.default

or from a service account file

https://google-auth.readthedocs.io/en/latest/reference/google.oauth2.service_account.html#google.oauth2.service_account.Credentials.from_service_account_file

You can pass the credentials object to the client constructor.

https://googlecloudplatform.github.io/google-cloud-python/stable/bigquery-client.html

I wonder if the helper methods such as from_service_account_json() should be removed in favor of using Google Auth credentials directly?

[cdchan]

Update: see my comment below.

---

@tswast cool, thanks! I was able to get

SCOPE = ('https://www.googleapis.com/auth/bigquery',
         'https://www.googleapis.com/auth/cloud-platform',
         'https://www.googleapis.com/auth/drive')
credentials, _ = google.auth.default(scopes=SCOPE)
client = bigquery.Client(credentials=credentials)

to work. Don't know why my subclass didn't before but this'll do.