Skip to content

Support INSERT in Google Sheets connector #15026

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 29, 2023

Conversation

sbernauer
Copy link
Member

@sbernauer sbernauer commented Nov 15, 2022

Description

Fixes #3866. Extracted parts out of #5011

This PR add support for INSERT statements in the Google Sheets connector

Non-technical explanation

Release notes

( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
(X) Release notes are required, with the following suggested text:

# Google Sheets connector
* Add support for the `INSERT` statement ({issue}`15026`)

@cla-bot cla-bot bot added the cla-signed label Nov 15, 2022
@github-actions github-actions bot added the docs label Nov 15, 2022
@ebyhr ebyhr self-requested a review November 15, 2022 09:55
@ebyhr
Copy link
Member

ebyhr commented Nov 16, 2022

Please separate a commit into INSERT and type mapping.

@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch from e30efd7 to 088ccb7 Compare November 16, 2022 13:04
@sbernauer
Copy link
Member Author

@ebyhr thanks for your fast and extensive review! All feedback should be adressed

@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch 2 times, most recently from ae699d9 to e73f7fc Compare November 16, 2022 14:21
@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch 5 times, most recently from 10942a6 to 48aadfa Compare November 17, 2022 09:49
@sbernauer
Copy link
Member Author

sbernauer commented Nov 17, 2022

@ebyhr many thanks again! Feedback should be addressed

@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch 2 times, most recently from e98b8a8 to 2887cfa Compare November 22, 2022 08:13
@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch from 2887cfa to eb22752 Compare December 22, 2022 12:33
@sbernauer
Copy link
Member Author

Rebased and fixed all the introduced conflicts. I'm going to look into creating a dedicated spreadsheet for every test invocation.

@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch from eb22752 to 1eec3fe Compare December 22, 2022 18:11
@sbernauer
Copy link
Member Author

I refactored all the Google sheet tests to:

  1. Create a dedicated temporary spreadsheet for every test run
  2. Add Sheets (tabs) containing test data to the Spreadsheet
  3. We can now happily insert or modify data as we won't effect any other running test
  4. Afterwards the temporary spreadsheet gets deleted. We have to use Drive API to delete the spreadsheet :/ Currently this fails with Access Not Configured. Drive API has not been used in project 1006290340336 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/drive.googleapis.com/overview?project=1006290340336 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry. @ebyhr do you have the permission to enable this API or do know someone who can do this? Currently the cleanup code is commented out.

@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch from 1eec3fe to 374e319 Compare December 22, 2022 18:17
@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch from 374e319 to 2c43486 Compare January 5, 2023 07:34
@sbernauer
Copy link
Member Author

Rebased to master. @ebyhr it would be really great if you could review this please

@ebyhr
Copy link
Member

ebyhr commented Feb 16, 2023

Can we extract a PR for the 2nd commit? I'm not convinced the internal format of a new type mapping.

@ebyhr
Copy link
Member

ebyhr commented Mar 8, 2023

Please rebase on master to resolve conflicts.

@glarrain-cdd
Copy link

@sbernauer Congrats on this PR.

Do you think it is going to be merged anytime soon?

Sorry to bother; it's just that it is critical for us. We'd give a hand with the code but we are Python, not Java, experts, and we aren't familiar enough with Trino internals.

@ebyhr
Copy link
Member

ebyhr commented May 16, 2023

it's just that it is critical for us

@glarrain-cdd Which feature (INSERT or type mapping) is critical for you?

@glarrain-cdd
Copy link

INSERT!

Thank you very much

@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch from 2c43486 to 0d14fb6 Compare May 19, 2023 14:35
@sbernauer
Copy link
Member Author

Hi @glarrain-cdd and @ebyhr,

sorry for the slow response time. The customer I started implementing this dropped the requirement for this feature. Nevertheless I would like to get it in, just for the community purpose.

I did rework my entire PR (there were some changes regarding the TableHandle in the meantime). Additionally I dropped the whole type mapping thing, so that we can focus entirely on the INSERTs.
Would be happy about any feedback!

One open point is enabling Google Drive API access for the Google serviceuser the Trino project uses, so that we can clean up the Spreadsheets we create in the test run. I can't do that, some maintainer needs to do that.

All the best,
Sebastian

@sbernauer sbernauer changed the title Support INSERT and type mapping in Google Sheets connector Support INSERT in Google Sheets connector May 19, 2023
@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch from 0d14fb6 to c4aae16 Compare June 6, 2023 06:57
@sbernauer
Copy link
Member Author

Rebased and fixed conflicts.
@ebyhr gentle reminder, a review would be great!

@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch from c4aae16 to 030e199 Compare June 27, 2023 17:36
@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch from 030e199 to efc1ee9 Compare June 28, 2023 07:56
@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch 2 times, most recently from efe7a20 to d942f2c Compare June 28, 2023 09:11
@sbernauer
Copy link
Member Author

@mosabua thanks for the docs review! Addressed your comments

@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch from d942f2c to 5e12b0a Compare June 28, 2023 19:46
Copy link
Member Author

@sbernauer sbernauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kokosing thanks for your review! Feedback should be addressed

Copy link
Member

@mosabua mosabua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs look good now

Copy link
Member

@kokosing kokosing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

% comments

@sbernauer sbernauer force-pushed the google-sheets-insert-squashed branch from 5e12b0a to 0da69b2 Compare June 29, 2023 14:51
@sbernauer
Copy link
Member Author

@kokosing thanks again, feedback adressed

@kokosing kokosing merged commit 83e01e0 into trinodb:master Jun 29, 2023
@kokosing
Copy link
Member

Thank you. Merged. I am sorry for any delays you have experienced. This is really nice contribution.

@github-actions github-actions bot added this to the 421 milestone Jun 29, 2023
@sbernauer
Copy link
Member Author

Thanks for the quick review after posting on Slack!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Support INSERT in Google Sheets connector
6 participants