Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No option to specify a custom nullMarker while using table.upload_from_file method #3449

Closed
HassanJaveed84 opened this issue May 23, 2017 · 5 comments
Assignees
Labels
api: bigquery Issues related to the BigQuery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@HassanJaveed84
Copy link

We have source data where Null values are represented by \N.

However, we're unable to use table.upload_from_file method as there is no parameter to specify a custom null marker(configuration.load.nullMarker).

@tseaver tseaver added api: bigquery Issues related to the BigQuery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. labels May 23, 2017
@tseaver
Copy link
Contributor

tseaver commented May 23, 2017

@HassanJaveed84 Thanks for the report. If you would care to prepare a pull request patching the table.py module, it would land more quickly. Looking at the code, it would need a new null_marker argument to Table.load_from_file, which would then get passed down to the _configure_job_metadata helper. The PR would also need to update one of the tests to demonstrate passing the value.

@HassanJaveed84
Copy link
Author

@tseaver I'll go ahead and create a PR, thanks!

@HassanJaveed84
Copy link
Author

@tseaver I tried following the instructions for local development on:
https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/CONTRIBUTING.rst

However, I'm unable to run the tests using the given instructions.
"To run all tests for google-cloud-python on a single Python version, run py.test from your development virtualenv "

Not sure about py.test ?

@dhermes
Copy link
Contributor

dhermes commented May 24, 2017

@HassanJaveed84 sorry CONTRIBUTING is very out of date (/cc @lukesneeringer).

Instead, install nox via:

pip install --upgrade nox-automation

then run nox in the bigquery directory:

$ cd path/to/git/repo/
$ cd bigquery/
$ nox -l  # list of tests
$ nox -s "unit_tests(python_version='3.6')"  # One test
$ nox  # All tests, though you probably don't want system tests

@HassanJaveed84
Copy link
Author

HassanJaveed84 commented May 29, 2017

Thanks @dhermes.
I'll let you know if I run into problems again.
I also found that client.load_table_from_storage does not accept a null_marker value.
I'll try to address both of these issues.

@lukesneeringer lukesneeringer self-assigned this Aug 9, 2017
planestepper pushed a commit to planestepper/google-cloud-python that referenced this issue Aug 9, 2017
@lukesneeringer lukesneeringer removed the priority: p2 Moderately-important priority. Fix may not be included in next release. label Aug 9, 2017
landrito pushed a commit to landrito/google-cloud-python that referenced this issue Aug 21, 2017
landrito pushed a commit to landrito/google-cloud-python that referenced this issue Aug 22, 2017
landrito pushed a commit to landrito/google-cloud-python that referenced this issue Aug 22, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the BigQuery API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

4 participants