Skip to content

BUG: Debug grouped quantile with NA values #33571

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from
Closed

BUG: Debug grouped quantile with NA values #33571

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
7778129
BUG: Debug grouped quantile with NA values
dsaxton Apr 15, 2020
1297438
Comment
dsaxton Apr 15, 2020
db39b0a
Fixup
dsaxton Apr 15, 2020
99ddd69
Merge remote-tracking branch 'upstream/master' into groupby-quantile
dsaxton Apr 17, 2020
f10e9dd
Better test
dsaxton Apr 17, 2020
0257f86
Remove
dsaxton Apr 17, 2020
e921701
Black
dsaxton Apr 17, 2020
727997d
Merge remote-tracking branch 'upstream/master' into groupby-quantile
dsaxton Apr 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fixup
  • Loading branch information
@dsaxton
dsaxton committed Apr 15, 2020
commit db39b0a2065aa0b324b290782fb118f2a5d67127
6 changes: 5 additions & 1 deletion pandas/_libs/groupby.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -780,7 +780,11 @@ def group_quantile(ndarray[float64_t] out,

# Get an index of values sorted by labels and then values,
# make sure missing labels sort to the back of the array
order = (values, np.where(labels == -1, labels.max() + 1, labels))
if labels.size:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the subsequent loop even account for missing labels in iteration? If not I wonder if cleaner to just remove rather than force sorting this way

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I think we'd also have to remove from values so that the two have the same shape for lexsort (that might break what's below but I'm not sure honestly)

labels_for_lexsort = np.where(labels == -1, labels.max() + 1, labels)
else:
labels_for_lexsort = labels
order = (values, labels_for_lexsort)
sort_arr = np.lexsort(order).astype(np.int64, copy=False)

with nogil:
Expand Down