refactor(bigquery): update code samples to use strings for table and dataset IDs #9495

emar-kar · 2019-10-18T10:58:32Z

Towards #8989
Closes #9499.

List of changes:

New samples:

client_query_add_column;
client_query_batch;
client_query_destination_table;
client_query_destination_table_cmek;
client_query_destination_table_legacy;
client_query_dry_run;
client_query_legacy_sql;
client_query_relax_column;
copy_table_cmek;
copy_table_multiple_source.

Comment lines in samples unified:

client_query;
copy_table;
load_table_dataframe.

DEPRECATED tests

test_create_routine;
test_update_routine;
test_get_routine;
test_delete_routine;
test_list_routines;
test_create_routine_ddl

All those tests were united in the test_routine_samples file!

…-python into new-samples

bigquery/samples/copy_table_multiple_source.py

bigquery/samples/copy_table_cmek.py

bigquery/samples/client_query_legacy_sql.py

bigquery/samples/client_query_destination_table_cmek.py

bigquery/samples/tests/test_client_query_dry_run.py

bigquery/samples/tests/test_client_query_batch.py

bigquery/samples/client_query_destination_table_cmek.py

bigquery/samples/tests/test_client_query_dry_run.py

bigquery/samples/tests/test_client_query_legacy_sql.py

…-python into new-samples

bigquery/docs/usage/queries.rst

The parameter can sometimes confuse new BigQuery developers. Since location autodetection now works pretty well, the parameter can be removed from code samples for better clarity, except where the samples want to explicitly demonstrate its usage.

googlebot · 2019-10-25T18:59:28Z

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

plamut · 2019-10-25T19:01:39Z

@googlebot I consent.

googlebot · 2019-10-25T19:01:48Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

bigquery/samples/copy_table_cmek.py

bigquery/samples/client_query_destination_table_legacy.py

bigquery/samples/client_query_legacy_sql.py

bigquery/samples/client_query_destination_table.py

bigquery/samples/client_query_destination_table_cmek.py

bigquery/samples/client_query_batch.py

bigquery/samples/client_query_relax_column.py

tswast · 2019-11-27T23:38:25Z

bigquery/samples/tests/test_copy_table_multiple_source.py

+
+    copy_table_multiple_source.copy_table_multiple_source(
+        client, random_table_id, table_ids
+    )


Let's call dest_table = client.get_table(random_table_id) and then assert dest_table.num_rows > 0 so that we know that the job did what we expected it to do.

bigquery/samples/tests/test_routine_samples.py

tswast · 2019-11-27T23:43:09Z

bigquery/samples/tests/test_client_query_destination_table_legacy.py



-def test_get_routine(capsys, client, routine_id):
+def test_client_query_destination_table_legacy(capsys, client, table_id):


Use random_table_id since we want to create a new table in the actual sample.

The table_id creates a table before the tests run.

google-cloud-python/bigquery/samples/tests/conftest.py

Line 76 in 9360057

table = client.create_table(table)

tswast · 2019-11-27T23:53:57Z

bigquery/samples/tests/test_client_query_dry_run.py

-    assert routine.body == "x * 4"
+    query_job = client_query_dry_run.client_query_dry_run(client)
+    out, err = capsys.readouterr()
+    assert "This query will process 0 bytes." not in out


I got confused by this test.

I'd very much like to see a positive assertion that a non-zero number is present. not in could happen for many different reasons.

Suggested change

assert "This query will process 0 bytes." not in out

assert "This query will process" in out

assert query_job.total_bytes_processed > 0

emar-kar added 2 commits October 16, 2019 18:04

refactor(bigquery): update code samples

b22b4ed

kms_key_name update

44c3e65

emar-kar added the api: bigquery Issues related to the BigQuery API. label Oct 18, 2019

googlebot added the cla: yes This human has signed the Contributor License Agreement. label Oct 18, 2019

emar-kar added 2 commits October 18, 2019 13:59

refactor(bigquery): kms_key_name update

45d90de

Merge branch 'new-samples' of https://github.com/q-logic/google-cloud…

5631419

…-python into new-samples

emar-kar changed the title ~~New samples~~ refactor(bigquery): update code samples to use strings for table and dataset IDs Oct 18, 2019

emar-kar requested a review from tswast October 18, 2019 11:01

emar-kar marked this pull request as ready for review October 18, 2019 11:01

emar-kar requested a review from a team October 18, 2019 11:01

emar-kar added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 18, 2019

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 18, 2019

tswast added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 18, 2019

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Oct 18, 2019

tswast requested changes Oct 18, 2019

View reviewed changes

plamut mentioned this pull request Oct 19, 2019

docs(bigquery): remove location parameter from samples #9500

Closed

4 tasks

samples update

5af72c8

emar-kar commented Oct 21, 2019

View reviewed changes

emar-kar added 4 commits October 21, 2019 13:43

docs session fix

1ab9eb7

docs session fix

14fbab4

Merge branch 'new-samples' of https://github.com/q-logic/google-cloud…

c1c2c7c

…-python into new-samples

update docs

f15033c

emar-kar commented Oct 21, 2019

View reviewed changes

bigquery/docs/usage/queries.rst Show resolved Hide resolved

emar-kar and others added 4 commits October 24, 2019 13:32

assertion update with re

bfa3ec2

Merge branch 'master' into new-samples

9326f90

Pass location where auto-detection not supported

04555f9

googlebot added cla: no This human has *not* signed the Contributor License Agreement. and removed cla: yes This human has signed the Contributor License Agreement. labels Oct 25, 2019

googlebot added cla: yes This human has signed the Contributor License Agreement. and removed cla: no This human has *not* signed the Contributor License Agreement. labels Oct 25, 2019

emar-kar mentioned this pull request Nov 4, 2019

BigQuery: refactor bigquery_query_params_* code samples to pass query_parameters to QueryJobConfig constructor #9596

Closed

3 tasks

tswast requested changes Nov 5, 2019

View reviewed changes

update QueryJobConfig

b2cb5a1

emar-kar added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 8, 2019

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Nov 8, 2019

emar-kar added 2 commits November 11, 2019 16:41

Merge branch 'master' into new-samples

08a1a2f

unify undelete_table sample

1c135d7

tswast self-requested a review November 13, 2019 01:17

tswast requested changes Nov 27, 2019

View reviewed changes

emar-kar added 2 commits November 28, 2019 10:42

refactor: update test files with the new conditions

1eb6a25

refactor: remove extra blank line in test files

16dd3f8

tswast self-requested a review December 2, 2019 15:34

tswast approved these changes Dec 2, 2019

View reviewed changes

tswast merged commit 371c4f2 into googleapis:master Dec 2, 2019

emar-kar deleted the new-samples branch December 3, 2019 08:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(bigquery): update code samples to use strings for table and dataset IDs #9495

refactor(bigquery): update code samples to use strings for table and dataset IDs #9495

emar-kar commented Oct 18, 2019 •

edited by plamut

Loading

googlebot commented Oct 25, 2019

plamut commented Oct 25, 2019

googlebot commented Oct 25, 2019

tswast Nov 27, 2019

tswast Nov 27, 2019

tswast Nov 27, 2019



		def test_get_routine(capsys, client, routine_id):
		def test_client_query_destination_table_legacy(capsys, client, table_id):

	assert "This query will process 0 bytes." not in out
	assert "This query will process" in out
	assert query_job.total_bytes_processed > 0

refactor(bigquery): update code samples to use strings for table and dataset IDs #9495

refactor(bigquery): update code samples to use strings for table and dataset IDs #9495

Conversation

emar-kar commented Oct 18, 2019 • edited by plamut Loading

List of changes:

New samples:

Comment lines in samples unified:

DEPRECATED tests

googlebot commented Oct 25, 2019

plamut commented Oct 25, 2019

googlebot commented Oct 25, 2019

tswast Nov 27, 2019

Choose a reason for hiding this comment

tswast Nov 27, 2019

Choose a reason for hiding this comment

tswast Nov 27, 2019

Choose a reason for hiding this comment

emar-kar commented Oct 18, 2019 •

edited by plamut

Loading