Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DataFrame] Implement df.merge #1964

Merged
merged 13 commits into from
May 2, 2018
Merged

[DataFrame] Implement df.merge #1964

merged 13 commits into from
May 2, 2018

Conversation

devin-petersohn
Copy link
Member

@devin-petersohn devin-petersohn commented Apr 28, 2018 •
edited
Loading

Implement the DataFrame.merge method that does a Database style join on common columns. Depends on #1937.

TODO:

  • Add tests for merge

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5107/
Test FAILed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5114/
Test PASSed.

p-yang
p-yang suggested changes Apr 29, 2018
View reviewed changes
python/ray/dataframe/dataframe.py Outdated
self._col_metadata = col_metadata
else:
assert columns is not None, \
"Columns must be passed without col_metadata"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"If col_metadata is None, columns must be passed in"

python/ray/dataframe/dataframe.py Outdated
else:
repartitioned_right = right._block_partitions

left_cols = ray.put(self.columns)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Put this before column merge

p-yang
p-yang reviewed Apr 30, 2018
View reviewed changes
python/ray/dataframe/dataframe.py Outdated

# TODO ERROR CHECK
if on is not None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perform error checking before defining new_columns so users don't get a local error followed by a remote error.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5126/
Test PASSed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5130/
Test FAILed.

@devin-petersohn
Copy link
Member Author

Jenkins, test this please

@devin-petersohn
Copy link
Member Author

Jenkins, retest this please.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/5137/
Test PASSed.

@robertnishihara robertnishihara merged commit 7c1d569 into ray-project:master May 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@p-yang p-yang p-yang requested changes

@robertnishihara robertnishihara robertnishihara approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@devin-petersohn @AmplabJenkins @robertnishihara @p-yang