Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connections: Add Spark option for R and Postgre for Python #6428

Merged
merged 7 commits into from
Feb 27, 2025
Merged

Conversation

dfalbel
Copy link
Contributor

@dfalbel dfalbel commented Feb 21, 2025

Adresses: #6186
Adresses: #6184
Addresses: #6188
Addresses: #6453

Adds options for connecting to Spark with R and Postgres with Python.
Also adds icons for connections, so they are nicely rendered in the new connections modal:

CleanShot 2025-02-21 at 13 16 59

I'm not sure though what's the correct way to make sure we have the correct copyright notice.

Also fixes a few bugs:

Release Notes

New Features

  • Added Spark and PostgreSQL as options when creating new connections using the Connections Pane.

Bug Fixes

  • N/A

QA Notes

Not sure if it's worth testing that generated code is valid for now. Setting up both Spark and Postgre locally can be tricky.

For spark, the following should work:

# Run once
install.packages("sparklyr")
sparklyr::spark_install()

# Connect to Spark local
library(sparklyr)
sc <- spark_connect(master = "local")

For PostgreSQL, it's also not very easy to spin up a local DB for testing. The following worked for me:

brew services start postgresql

Happy to help with the set up :)

CI failures are related to #6496

@:connections

Copy link

github-actions bot commented Feb 21, 2025

E2E Tests 🚀
This PR will run tests tagged with: @:critical @:connections

readme  valid tags

jmcphers
jmcphers previously approved these changes Feb 26, 2025
Copy link
Collaborator

@jmcphers jmcphers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Co-authored-by: Jonathan <jonathan@rstudio.com>
Signed-off-by: Daniel Falbel <dfalbel@gmail.com>
@dfalbel dfalbel merged commit 3ef420c into main Feb 27, 2025
19 of 29 checks passed
@dfalbel dfalbel deleted the feature/spark branch February 27, 2025 01:18
@github-actions github-actions bot locked and limited conversation to collaborators Feb 27, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants