-
Notifications
You must be signed in to change notification settings - Fork 893
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[KED-1876] Allow GBQTableDataSet to optionally accept a sql query to load data #443
[KED-1876] Allow GBQTableDataSet to optionally accept a sql query to load data #443
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is really close to ready, thank you for the contribution! 🥳
Some minor comments that you need to address then I think we're good to go. Happy to support you whilst addressing the comments as well!
…to feature/modified-gbqtabledataset-to-accept-custom-queries
…es' of https://github.com/ajb7/kedro into feature/modified-gbqtabledataset-to-accept-custom-queries
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome work! 🎉
loaded_data = gbq_dataset.load() | ||
|
||
mocked_read_gbq.assert_called_once_with( | ||
project_id=PROJECT, credentials=None, query=load_args["query"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’d check query=“Select 1” explicitly here but up to you!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM 👍
Thanks again @ajb7 |
Thank you Kedro Team!!! 😄 ⭐ |
Description
Modify current implementation of
GBQTableDataSet
to accept custom queries to be executed in GBQ to load custom results.Addresses: #442
Development notes
Modified the
_load()
method inGBQTableDataSet
to accept custom queries passed viaload_args
incatalog.yml
. Eventually passing the arguments inpandas.read_gbq()
Checklist
RELEASE.md
fileNotice
[X ] I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":
I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorised to submit this contribution on behalf of the original creator(s) or their licensees.
I certify that the use of this contribution as authorised by the Apache 2.0 license does not violate the intellectual property rights of anyone else.