Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQuery: expose public helper method to convert a list of schema fields to/from a list of schema dictionaries (JSON) #3723

Closed
tswast opened this issue Aug 3, 2017 · 4 comments
Assignees
Labels
api: bigquery Issues related to the BigQuery API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@tswast
Copy link
Contributor

tswast commented Aug 3, 2017

I've received some feedback asking how to take a schema from the client library and save it to a JSON file. One reason to do this is the bq command-line tool accepts a schema file, formatted like

[
  {"name": "name", "type": "string", "mode": "required"},
  {"name": "gender", "type": "string", "mode": "nullable"},
  {"name": "count", "type": "integer", "mode": "required"}
]

Note: this format is the same as the API representation.

It would be great if our client libraries could read/write in this format.

@tswast tswast added api: bigquery Issues related to the BigQuery API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. labels Aug 3, 2017
@tseaver
Copy link
Contributor

tseaver commented Aug 3, 2017

We already have non-public helper methods which render / parse schemas to / from a format like:

{"fields": [
  {"name": "name", "type": "string", "mode": "required"},
  {"name": "gender", "type": "string", "mode": "nullable"},
  {"name": "count", "type": "integer", "mode": "required"}
]}

It would be pretty trivial to adapt them to skip / overlook the missing fields key. How would you envision this for the library API surface? Methods on the Table class to load / store the schema to a file? What about schema for e.g. query jobs?

@tswast
Copy link
Contributor Author

tswast commented Aug 3, 2017

There are two main use cases that I've heard about

  • at table creation/load time, being able to load schema from a file.
  • after a query so that they can serialize the schema from the stats to then use to create other table.

@tswast
Copy link
Contributor Author

tswast commented Aug 3, 2017

Since this is something that would act on lists, it doesn't really correspond to an existing resource representation that we have. Do we put static helper functions in the top-level google.cloud.apiname package, generally?

@tswast
Copy link
Contributor Author

tswast commented Aug 3, 2017

Going through the issues I see that this is a duplicate of #3419.

Closing this issue in favor of that one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. priority: p2 Moderately-important priority. Fix may not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

2 participants