Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add usage docs for jobs: query and load. #1014

Merged
merged 5 commits into from
Jul 24, 2015
Merged

Add usage docs for jobs: query and load. #1014

merged 5 commits into from
Jul 24, 2015

Conversation

tseaver
Copy link
Contributor

@tseaver tseaver commented Jul 23, 2015

No description provided.

@tseaver tseaver added the api: bigquery Issues related to the BigQuery API. label Jul 23, 2015
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Jul 23, 2015

- Load data into a table
- Run a query against data in one or more tables
- Extrat data from a table

This comment was marked as spam.

This comment was marked as spam.

Addresses feedback from @dhermes on #1014.
Addresses feedback from @dhermes in #1014.
>>> job = table.load_from_storage(bucket_name='bucket-name',
... object_name='object-prefix*',
... source_format='CSV',
... skip_leading_rows=1,

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

@dhermes
Copy link
Contributor

dhermes commented Jul 23, 2015

Just two things left

  • "Should the name not be object_names or object_expression or something more generic?"
  • It seems the idiom of polling is ubiquitous. We should consider a class or method to aid in this process

@tseaver
Copy link
Contributor Author

tseaver commented Jul 24, 2015

"Should the name not be object_names or object_expression or something more generic?"

Per Loading Data into BigQuery, the name is used to fill a URI in the form gs://<bucket>/<object>.

Per https://cloud.google.com/bigquery/docs/reference/v2/jobs#configuration.load, that URE can have the glob character only at the end. That value is populated into a repeatable JSON field, configuration.load.sourceUris. We want to support multiple values, any of which might be globbed.

It seems the idiom of polling is ubiquitous. We should consider a class or method to aid in this process.

Maybe so. There might even be a call to use it in pubsub (for a pull-mode subscription) or storage (does it have pull-mode change notification?). I don't really see it as a blocker for this PR, though.

@dhermes
Copy link
Contributor

dhermes commented Jul 24, 2015

Pull idiom not a blocker for this PR, but maybe we should file a tracking issue?

As for the rest, the question is about the appropriate name for the variable, not about the feature. I already understood the URI from your previous comment.

object_name does not capture the possibility of a group of files (via a glob) so I'm proposing we consider object_names or object_expression as the variable name.

@tseaver
Copy link
Contributor Author

tseaver commented Jul 24, 2015

object_names doesn't work -- it implies a sequence. object_name_glob, maybe?

@dhermes
Copy link
Contributor

dhermes commented Jul 24, 2015

It's rightly a regex, yes?

@tseaver
Copy link
Contributor Author

tseaver commented Jul 24, 2015

Nope (there is no preceding ., for instance). It really is a prefix-only glob pattern.

@dhermes
Copy link
Contributor

dhermes commented Jul 24, 2015

OK. SGTM.

Change it to your choice of more descriptive variable name and LGTM.

@tseaver
Copy link
Contributor Author

tseaver commented Jul 24, 2015

@dhermes 48eef92 changes the argument name to object_name_glob.

@dhermes
Copy link
Contributor

dhermes commented Jul 24, 2015

SGTM. We're all good to merge then, right?

@tseaver
Copy link
Contributor Author

tseaver commented Jul 24, 2015

Yup

tseaver added a commit that referenced this pull request Jul 24, 2015
Add usage docs for jobs:  query and load.
@tseaver tseaver merged commit fd6c48a into googleapis:master Jul 24, 2015
@tseaver tseaver deleted the bigquery-devx-jobs-query_load branch July 27, 2015 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants