Skip to content

Revert BUG-24212 fix usage of Index.take in pd.merge #24904

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 24, 2019
Merged

Revert BUG-24212 fix usage of Index.take in pd.merge #24904

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
634030b
Revert BUG-24212 fix usage of Index.take in pd.merge
TomAugspurger Jan 24, 2019
fedcec2
test 0.23.4 output
TomAugspurger Jan 24, 2019
9c45869
added note about buggy test
TomAugspurger Jan 24, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
added note about buggy test
  • Loading branch information
@TomAugspurger
TomAugspurger committed Jan 24, 2019
commit 9c458698b9014b8313d1a3452c098d5ad050a2dc
4 changes: 3 additions & 1 deletion pandas/tests/reshape/merge/test_merge.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -961,7 +961,9 @@ def test_merge_on_index_with_more_values(self, how):
assert_frame_equal(result, expected)

def test_merge_right_index_right(self):
# https://github.com/pandas-dev/pandas/issues/24897
# Note: the expected output here is probably incorrect.
# See https://github.com/pandas-dev/pandas/issues/17257 for more.
# We include this as a regression test for GH-24897.
left = pd.DataFrame({'a': [1, 2, 3], 'key': [0, 1, 1]})
right = pd.DataFrame({'b': [1, 2, 3]})

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can others double-check this expected output? This is what 0.23.4 / this PR gives. I'm a bit confused about the duplicate 2 in the index.

In [5]: expected
Out[5]:
     a  key  b
0  1.0    0  1
1  2.0    1  2
2  3.0    1  2
2  NaN    2  3

I may have expected something like

In [11]: pd.merge(left, right.rename_axis('key').reset_index(), on='key', how='right').set_index('key', drop=False)
Out[11]:
       a  key  b
key
0    1.0    0  1
1    2.0    1  2
1    3.0    1  2
2    NaN    2  3

where the 1 is duplicated. Or am I missing something?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I was first looking at the 'b' column (where the duplicate 2 is of course normal), but the index :)

I assume this is because with how='right and right_index=True, we use the index of the right for the index of the final dataframe (so the 'key' column corresponds to the index in the result).

Not sure this is fully correct though. I don't know if we have somewhere a specification of what the resulting index of a merge should be.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the 'key' column corresponds to the index in the result

and again looking wrongly, because what I describe is your expected result, not the actual ...

Yes, so that certainly looks wrong

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:)

I may just add a comment that this test should probably fail as well, and open a new issue about it, since this is the behavior on 0.23.4.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, a comment would be good

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm this looks vaguely similar to #17257

Expand Down