Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove config property hive.gcs.scheme #11235

Closed
wants to merge 3 commits into from
Closed

Remove config property hive.gcs.scheme #11235

wants to merge 3 commits into from

Conversation

majetideepak
Copy link
Collaborator

@majetideepak majetideepak commented Oct 11, 2024 •
edited
Loading

The GCS adapter expects two configs hive.gcs.scheme value for the scheme and hive.gcs.endpoint value without the scheme.
However, in practice, the endpoint value contains the scheme. We can remove hive.gcs.scheme to be consistent.
See Trino for example https://trino.io/docs/current/admin/fault-tolerant-execution.html#google-cloud-storage

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 11, 2024
@netlify Netlify
Copy link

netlify bot commented Oct 11, 2024 •
edited
Loading

✅ Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 922a3ea
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/670f92a0c854a200085a45cd

@majetideepak
Copy link
Collaborator Author

@tigrux any feedback on this? Thanks!

@majetideepak majetideepak mentioned this pull request Oct 11, 2024
Open
tigrux
tigrux approved these changes Oct 11, 2024
View reviewed changes
Copy link
Contributor

@tigrux tigrux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!
It totally makes sense.

@majetideepak
Copy link
Collaborator Author

I tested this change locally. It works!

@majetideepak majetideepak added the ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall label Oct 11, 2024
@facebook-github-bot
Copy link
Contributor

@kgpai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

czentgr
czentgr reviewed Oct 15, 2024
View reviewed changes
Copy link
Collaborator

@czentgr czentgr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit, lets make sure the doc is matching the (new) behavoir and leaves no questions.

velox/docs/configs.rst Outdated
@@ -633,10 +633,6 @@ Each query can override the config by setting corresponding query session proper
- string
-
- The GCS storage endpoint server.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe should mention that this is a URI that includes the scheme rather than just the server? This PR changes the behavior of this variable. Before it did require a scheme-less hostname (+port).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I added the endpoint default as https://storage.googleapis.com since it is the most common.
I also updated the doc.

@majetideepak
Copy link
Collaborator Author

@kgpai can you please reimport?

czentgr
czentgr approved these changes Oct 16, 2024
View reviewed changes
Copy link
Collaborator

@czentgr czentgr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@facebook-github-bot
Copy link
Contributor

@kgpai merged this pull request in 9d4cfb2.

@conbench-facebook conbench-facebook
Copy link

Conbench analyzed the 1 benchmark run on commit 9d4cfb25.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

@majetideepak majetideepak mentioned this pull request Oct 17, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tigrux tigrux tigrux approved these changes

@czentgr czentgr czentgr approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged ready-to-merge PR that have been reviewed and are ready for merging. PRs with this tag notify the Velox Meta oncall
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@majetideepak @facebook-github-bot @tigrux @czentgr